-- Logs begin at Mon 2022-09-19 09:44:51 UTC, end at Mon 2022-09-19 11:03:58 UTC. -- Sep 19 10:16:22.453524 dv-cinder-858275-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 19 10:16:22.465909 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 19 10:16:22.467933 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 19 10:16:22 2022] *** Sep 19 10:16:22.467933 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 19 10:16:22.467933 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 19 10:16:22.467933 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: nodename: dv-cinder-858275-1-iscsi Sep 19 10:16:22.467933 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: machine: x86_64 Sep 19 10:16:22.467933 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: clock source: unix Sep 19 10:16:22.467933 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: pcre jit disabled Sep 19 10:16:22.467933 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: detected number of CPU cores: 4 Sep 19 10:16:22.467933 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: current working directory: / Sep 19 10:16:22.467933 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: detected binary path: /usr/bin/uwsgi-core Sep 19 10:16:22.468598 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: your processes number limit is 47680 Sep 19 10:16:22.468598 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: your memory page size is 4096 bytes Sep 19 10:16:22.468598 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: detected max file descriptor number: 2048 Sep 19 10:16:22.468598 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: lock engine: pthread robust mutexes Sep 19 10:16:22.468598 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: thunder lock: enabled Sep 19 10:16:22.468598 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 19 10:16:22.468598 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 19 10:16:22.506731 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: Python main interpreter initialized at 0x558c032b39c0 Sep 19 10:16:22.506731 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: python threads support enabled Sep 19 10:16:22.506731 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: your server socket listen backlog is limited to 100 connections Sep 19 10:16:22.506731 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: your mercy for graceful operations on workers is 90 seconds Sep 19 10:16:22.507030 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: mapped 403077 bytes (393 KB) for 2 cores Sep 19 10:16:22.507135 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: *** Operational MODE: preforking *** Sep 19 10:16:22.507402 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: *** uWSGI is running in multiple interpreter mode *** Sep 19 10:16:22.507402 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: spawned uWSGI master process (pid: 81551) Sep 19 10:16:22.507577 dv-cinder-858275-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 19 10:16:22.509235 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: spawned uWSGI worker 1 (pid: 81562, cores: 1) Sep 19 10:16:22.509235 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: spawned uWSGI worker 2 (pid: 81563, cores: 1) Sep 19 10:16:22.509235 dv-cinder-858275-1-iscsi devstack@keystone.service[81551]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 19 10:16:26.860440 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 10:16:26.860440 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: from cryptography.utils import int_from_bytes Sep 19 10:16:26.861175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 10:16:26.861175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: from cryptography.utils import int_from_bytes Sep 19 10:16:26.866073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 10:16:26.866073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: from cryptography.utils import int_from_bytes Sep 19 10:16:26.866558 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 10:16:26.866558 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: from cryptography.utils import int_from_bytes Sep 19 10:16:26.918502 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 19 10:16:26.918691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 19 10:16:26.918874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] command line args: [] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 19 10:16:26.919000 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 19 10:16:26.919146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 19 10:16:26.919229 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 19 10:16:26.919329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ================================================================================ {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 19 10:16:26.919417 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] command line args: [] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 19 10:16:26.919587 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] admin_token = **** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.919678 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 19 10:16:26.919788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ================================================================================ {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 19 10:16:26.919867 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] config_dir = [] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.920011 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.920096 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] admin_token = **** {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.920197 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] config_source = [] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.920288 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] config_dir = [] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.920368 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] debug = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.920447 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.920566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.920870 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] config_source = [] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.920870 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] debug = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.921120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.921233 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.921363 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.921467 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.921574 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.921685 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.921788 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.921890 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.921994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] list_limit = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.922103 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.922216 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] log_config_append = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.922313 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] list_limit = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.922407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.922501 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] log_config_append = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.922609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] log_dir = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.922705 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.922811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] log_file = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.922893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] log_dir = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.922893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] log_file = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.923090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.923206 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.923454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.923673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.923760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.923874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.923956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.924081 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.924239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.924364 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.924440 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.924602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.924685 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.924786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.924918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.925035 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.925201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.925309 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.925397 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.925550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.925671 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.925786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.925864 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.926025 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.926146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.926264 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.926344 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.926516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.926766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.926852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.926978 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] publish_errors = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.927062 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.927174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.927404 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.927524 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.927614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.927707 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] publish_errors = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.927816 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.927942 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.928094 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.928170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.928261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] standard_threads = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.928335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.928488 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.928579 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.928729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.928809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.928904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.929025 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] standard_threads = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.929180 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] use_journal = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.929269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] use_json = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.929343 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.929414 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] use_stderr = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.929523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.929726 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] use_syslog = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.929837 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.929953 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.930040 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] use_journal = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.930040 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] use_json = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.930208 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.930291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] use_stderr = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.930369 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.930450 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] use_syslog = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.930568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.930644 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 10:16:26.930788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.930869 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.931036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.931194 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.931269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.931340 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.931505 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.931592 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.931739 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.931826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.931984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.932064 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.932231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.932310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.932469 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.932554 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.932700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.932778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.932944 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.933024 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.933203 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.933286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.933451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.933532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.933701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.933942 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.934028 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.934192 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.934278 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.934444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.934525 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.934687 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.934774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.934945 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.935029 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.935176 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.935254 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.935425 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.935517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.935667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.935825 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.935924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.936004 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.936182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.936272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.936414 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.936493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.936644 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.936730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.936874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.937031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.941524 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.941652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.941752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.941826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.941826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.941964 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.941964 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.942094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.942164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.942234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.942309 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.942392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.942392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.942526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.942526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.942661 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.942731 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.942731 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.942876 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.942948 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.943018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.943087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.943165 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.943244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.943317 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.943393 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.943470 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.943581 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.943688 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.943786 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.943882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.943985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.944082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.944211 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.944323 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.944422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.944519 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.944649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.944743 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.944864 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.944953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.945120 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.945201 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.945274 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.945392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.945479 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.945563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.945645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.945719 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.945846 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.945920 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.945993 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.946118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.946243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.946321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.946452 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.946563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.946625 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.946738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.946862 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.946941 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.947097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.947174 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.947247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.947366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.947451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.947533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.947607 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.947733 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.947816 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.947891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.948021 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.948144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.948228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.948308 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.948390 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.948464 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.948582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.948668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.948786 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.948893 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.948968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.949036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.949191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.949279 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.949354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.949433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.949505 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.949578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.949649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.949729 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.949804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.949876 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.949994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.950071 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.950143 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.950290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.950367 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.950367 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.950497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.950567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.950685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.950771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.950845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.950916 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.950987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.951057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.951128 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.951211 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.951296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.951370 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.951441 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.951513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.951585 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.951655 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.951735 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.951810 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.951881 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.951951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.952022 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.952094 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.952176 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.952250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.952321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.952393 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.952464 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.952524 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.952673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.952767 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.952935 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.953036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.953148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.953222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.953308 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.953385 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.953464 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.953537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.953609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.953799 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.954431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.954601 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.954682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.954838 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.954918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.955073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.955162 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.955311 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.955398 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.955544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.955719 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.955868 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.955955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.956031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.956103 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.956248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.956329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.956488 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.956579 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] auth.external = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.956745 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.956852 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.957010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.957113 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.957287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.957380 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.957545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] auth.external = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.957641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] auth.password = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.957756 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.957861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] auth.token = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.958058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.958141 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.958296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.958385 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.958531 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] auth.password = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.958612 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.958765 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] auth.token = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.958845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.959008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.959100 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.959248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.959333 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.959477 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.959565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.959712 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] credential.caching = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.959794 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.959956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.960037 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.960202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.960285 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.960433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.960522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.960670 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] credential.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.960759 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.960910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.960994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.961167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.961248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.961400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.961482 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.961652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.961731 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.961885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.961967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.962126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.962206 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.962373 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.962461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.962629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.962711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.962868 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.962949 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.963115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.963195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.963369 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.963454 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.963718 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.963819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] federation.caching = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.963946 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.964024 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.964106 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.964182 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.964342 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.964425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.964571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.964652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] federation.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.964816 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.964898 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.965053 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.965155 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.965323 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.965406 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.965555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.965636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.965804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.965891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.966038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.966127 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.966284 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.966367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.966518 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] identity.caching = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.966599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.966764 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.966845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.967004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.967086 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.967235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.967316 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.967468 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.967550 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.967711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.967794 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.967949 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.968030 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.968185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.968266 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.968428 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.968513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.968670 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.968751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.968904 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.968986 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.969156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.969240 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.969414 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.969620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.969701 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.969890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.969972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.970130 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.970214 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.970375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.970504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.970606 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.970687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.970842 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.970942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.971091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.971173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.971337 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.971418 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.971567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.971648 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.971803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.971884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.972033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.972116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.972268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.972350 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.972505 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.972586 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.972741 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.972823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.972983 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.973108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.973235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.975910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.975910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.976202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.976280 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.976353 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.976426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.976498 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.976580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.976656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.976736 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.976820 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.976895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.976968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.977049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.977156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.977230 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.977303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.977303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.977435 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.977525 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.977600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.977673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.977744 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.977744 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.977875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.977955 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.978029 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.978101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.978172 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.978244 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.978316 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.978396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.978469 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.978469 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.978601 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.978601 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.978731 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.978811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.978885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.978885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.978991 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.979049 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.979120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.979200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.979318 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.979398 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.979471 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.979544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.979617 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.979699 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.979774 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.979847 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.979926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.979988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.980060 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.980141 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.980221 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.980295 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.980367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.980439 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.980518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.980601 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.980676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.980748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.980821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.980900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.981001 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.981163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.981275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.981395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.981519 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.981642 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.981760 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.981888 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.982000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.982133 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.982238 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.982378 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.982467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.982623 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.982704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.982853 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.983001 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.983101 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.983191 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.983360 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.983454 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.983594 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.983676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.983830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.983921 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.984065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.984147 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.984311 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.984392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.984555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.984643 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.984790 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.984871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.985046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.985180 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.985308 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.985403 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.985558 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.985640 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.985793 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.985881 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.986029 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.986184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.986262 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.986335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.986707 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.986812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.986890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.986973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.987050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.987123 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.987196 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.987269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.987425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.987520 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] resource.caching = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.987674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.987756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.987919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.988000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.988152 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.988241 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.988393 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.988474 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.988637 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] resource.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.988717 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.988870 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.988958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.989145 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.989231 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.989383 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.989464 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.989622 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.989716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.989856 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] role.caching = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.989938 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.990098 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.990179 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] role.driver = sql {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.990327 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.990415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.990574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.990655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.990816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.990895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.991053 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] role.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.991142 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.991296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.991376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] role.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.991528 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.991608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.991775 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.991862 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.992021 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.992102 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.992267 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.992347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.992496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.992584 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.992740 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.992821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.993008 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:26.993149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.000614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.001374 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.001457 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.001534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.001534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.001671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.001760 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.001836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.001909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.001981 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.002054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.002126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.002206 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.002281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.002281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.002413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.002485 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.002557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.002637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.002711 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.002783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.002855 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.002926 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.003005 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.003087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.003162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.003234 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.003307 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.003379 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.003451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.003531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.003607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.003679 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.003750 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.003823 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.003894 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.003974 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.004048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.004121 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.004194 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.004266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.004344 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.004426 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] token.caching = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.004501 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.004573 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.004651 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.004725 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.004798 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.004878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.004952 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.005023 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.005118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.005193 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.005277 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] token.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.005354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.005427 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.005499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.005571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.005643 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.005724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.005798 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.005870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.005953 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.006031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.006104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.006184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.006259 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.006337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.006411 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.006483 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.006555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.006639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.006714 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.006794 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.006878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.006953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.007025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.007097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.007168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.connection = **** {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.007249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.007324 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.007396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.007474 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.007548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.007629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.007712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.007794 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.007794 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.007928 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.008000 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.008073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.connection = **** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.008144 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.008225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.008300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.008373 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.008445 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.008518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.008590 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.008740 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.008817 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.008890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.008963 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009035 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009274 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009394 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009453 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009512 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009583 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009647 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009706 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009824 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81563) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 19 10:16:27.009950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 10:16:27.009950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81562) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 19 10:16:27.037205 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.037392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.037484 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.037624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.037726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.037851 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.045909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.046050 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.074269 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.074449 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.074729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.074961 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.133941 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.134222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.183162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.183397 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.184521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.184754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.347620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.347787 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.347984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.348204 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.348359 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.348448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.348592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.348686 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.348814 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.349031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.349276 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.349487 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.376459 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.376687 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.376828 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.376914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.377069 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.377157 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.377306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.377392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.377527 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.377611 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.377745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.377838 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.377961 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.378173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.388979 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.389201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.405992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.406122 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.406207 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.406315 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.406431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.406542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.406645 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.406762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.406847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.407604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.407697 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.407821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.407906 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81562) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.408034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81563) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 10:16:27.425453 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.425646 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 10:16:27.425987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.426156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.426240 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 10:16:27.426377 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 10:16:27.426461 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.426697 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.426788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.426903 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 10:16:27.426985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.427176 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.427258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.427436 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.427756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.427965 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.436697 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.436978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.437281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.437568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.437736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.438014 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.438321 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.438604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.444493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.445646 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81563) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.447389 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.447676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.447982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.448253 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.448493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81563) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.448577 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.448776 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.448859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.449046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.449161 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.449346 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.449430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.449626 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.449721 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.449898 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.449982 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.450167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.450248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.450439 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.450707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.450977 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.451241 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.473676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.475055 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.475153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.475406 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.476501 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81563) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.476783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.480038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.480322 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.480600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.481479 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81563) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.481763 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.482043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.486928 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.487219 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.488449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81563) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.488735 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.492805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.493127 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.493422 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.493711 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.493991 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.494292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.494381 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81563) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.494552 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.494753 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.494847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.495044 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.495331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.495609 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.495886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.496156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.496425 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.510259 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.510539 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.511749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.517401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.517484 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.517563 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81563) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.518116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.518414 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.519813 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.520107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.522595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.524322 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81563) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.525731 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.527688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81563) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.529207 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.529510 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.529801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.530087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.530376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.530678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.531033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.531331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.531616 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.532493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.532795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.533102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.549350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81563) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.549642 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.549911 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.550177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.551022 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.551317 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.551591 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.551859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.556968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.558586 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.558686 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.560095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.560196 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.560371 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.561704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.561991 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.563018 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81563) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.563448 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.563730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.564020 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.564411 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.564731 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.565028 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.565338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.565639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.566030 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.575443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81563) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.575723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.580455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.580455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.580455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.580455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.580688 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.580763 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.580850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.580850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.580850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.580850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.580850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.580850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.593053 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81563) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.595142 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.596796 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81563) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.598165 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.599752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.600939 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.601277 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.602407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.602757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81563) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.603037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.603428 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.603870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.604156 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.604554 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.609780 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.610912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.611275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81563) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.612363 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.614142 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.614538 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.614818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.615098 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.615246 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.615637 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.615927 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.616206 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.622882 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81563) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.623170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.623453 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.623965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.624257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.624542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.630892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81563) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.631179 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.631455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.631727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.631929 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81562) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 10:16:27.632015 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.632216 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.632300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.632499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.632592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.632770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.632854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.633116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.638749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.638852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.638931 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.639006 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.639080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.639080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.639225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.639302 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.639376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.639451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81563) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.639525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.639525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.639525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.639525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.639525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81562) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 10:16:27.669171 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558c032b39c0 pid: 81563 (default app) Sep 19 10:16:27.669910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558c032b39c0 pid: 81562 (default app) Sep 19 10:16:27.672506 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198bdb9c-4da5-493d-9e48-c6424af05ea6 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:27.672794 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198bdb9c-4da5-493d-9e48-c6424af05ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:27.673082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198bdb9c-4da5-493d-9e48-c6424af05ea6 None None] PATH_INFO: `/v3` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:27.677209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Mon Sep 19 10:16:27 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:35.178675 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35ebaa-6c95-4b80-a326-c9d8a051ebc5 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:35.179163 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35ebaa-6c95-4b80-a326-c9d8a051ebc5 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:35.179243 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35ebaa-6c95-4b80-a326-c9d8a051ebc5 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:35.183482 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:16:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:16:35.189031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74340a-9e6b-4322-a48e-26fcf6caa515 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:35.189519 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74340a-9e6b-4322-a48e-26fcf6caa515 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:35.189619 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74340a-9e6b-4322-a48e-26fcf6caa515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:35.259210 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: INFO dbcounter [None req-9f74340a-9e6b-4322-a48e-26fcf6caa515 None None] Registered counter for database keystone Sep 19 10:16:35.277999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG oslo_db.sqlalchemy.engines [None req-9f74340a-9e6b-4322-a48e-26fcf6caa515 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81563) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 10:16:35.284614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writer thread running {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 10:16:35.462039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG passlib.handlers.bcrypt [None req-9f74340a-9e6b-4322-a48e-26fcf6caa515 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81563) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 19 10:16:35.462346 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG passlib.handlers.bcrypt [None req-9f74340a-9e6b-4322-a48e-26fcf6caa515 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81563) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 19 10:16:35.520716 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-9f74340a-9e6b-4322-a48e-26fcf6caa515 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:16:35.522327 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-9f74340a-9e6b-4322-a48e-26fcf6caa515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:16:35.647572 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:16:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 19 10:16:36.118135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35ebaa-6c95-4b80-a326-c9d8a051ebc5 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:36.187577 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: INFO dbcounter [None req-8d35ebaa-6c95-4b80-a326-c9d8a051ebc5 None None] Registered counter for database keystone Sep 19 10:16:36.209504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG oslo_db.sqlalchemy.engines [None req-8d35ebaa-6c95-4b80-a326-c9d8a051ebc5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81562) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 10:16:36.215550 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writer thread running {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 10:16:36.332726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35ebaa-6c95-4b80-a326-c9d8a051ebc5 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:36.336952 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc4d39-9ca4-4b7f-96a7-caebf165f86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:36.337605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc4d39-9ca4-4b7f-96a7-caebf165f86c admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:36.337886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc4d39-9ca4-4b7f-96a7-caebf165f86c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:36.338160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc4d39-9ca4-4b7f-96a7-caebf165f86c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:36.346223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cc4d39-9ca4-4b7f-96a7-caebf165f86c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:36.398556 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: WARNING keystone.common.rbac_enforcer.enforcer [None req-11cc4d39-9ca4-4b7f-96a7-caebf165f86c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 19 10:16:36.398844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cc4d39-9ca4-4b7f-96a7-caebf165f86c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:36.409271 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Mon Sep 19 10:16:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 10:16:36.415175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74340a-9e6b-4322-a48e-26fcf6caa515 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:36.431955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74340a-9e6b-4322-a48e-26fcf6caa515 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:36.436235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c3ccc-a4ae-4fb7-bd2d-4950b84b459d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:36.437356 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c3ccc-a4ae-4fb7-bd2d-4950b84b459d admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:36.437638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c3ccc-a4ae-4fb7-bd2d-4950b84b459d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:36.438527 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c3ccc-a4ae-4fb7-bd2d-4950b84b459d admin admin] PATH_INFO: `/v3/projects/1e8dac977aca46d48c891cd4c281d8cb` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:36.440107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c3ccc-a4ae-4fb7-bd2d-4950b84b459d admin admin] RBAC: Authorizing `identity:get_project(project_id=1e8dac977aca46d48c891cd4c281d8cb)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:36.493150 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: WARNING keystone.common.rbac_enforcer.enforcer [None req-813c3ccc-a4ae-4fb7-bd2d-4950b84b459d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 19 10:16:36.493433 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c3ccc-a4ae-4fb7-bd2d-4950b84b459d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:36.497916 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Mon Sep 19 10:16:36 2022] GET /identity/v3/projects/1e8dac977aca46d48c891cd4c281d8cb => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:38.604196 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe84516-c1df-4d1a-b7ae-91b156899daf None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:38.604731 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe84516-c1df-4d1a-b7ae-91b156899daf None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:38.604731 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe84516-c1df-4d1a-b7ae-91b156899daf None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:38.608828 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:16:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:16:38.614708 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6afc64-ae64-4599-bbb0-1f3b01b3b638 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:38.615071 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6afc64-ae64-4599-bbb0-1f3b01b3b638 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:38.615251 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6afc64-ae64-4599-bbb0-1f3b01b3b638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:38.635333 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-2e6afc64-ae64-4599-bbb0-1f3b01b3b638 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:16:38.636834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-2e6afc64-ae64-4599-bbb0-1f3b01b3b638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:16:38.644492 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:16:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 19 10:16:38.913836 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe84516-c1df-4d1a-b7ae-91b156899daf None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:38.929258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe84516-c1df-4d1a-b7ae-91b156899daf None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:38.933387 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c7039-10a7-4c65-865a-2464e8f76ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:38.933978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c7039-10a7-4c65-865a-2464e8f76ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:38.934255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c7039-10a7-4c65-865a-2464e8f76ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:38.934528 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c7039-10a7-4c65-865a-2464e8f76ef2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:38.942633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c7039-10a7-4c65-865a-2464e8f76ef2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:38.944386 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c7039-10a7-4c65-865a-2464e8f76ef2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:38.952911 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Mon Sep 19 10:16:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 10:16:38.958754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6afc64-ae64-4599-bbb0-1f3b01b3b638 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:38.967783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6afc64-ae64-4599-bbb0-1f3b01b3b638 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:38.972190 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a75994-c891-436f-9a11-d5cd0dd8f678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:38.972815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a75994-c891-436f-9a11-d5cd0dd8f678 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:38.973127 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a75994-c891-436f-9a11-d5cd0dd8f678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:38.973410 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a75994-c891-436f-9a11-d5cd0dd8f678 admin admin] PATH_INFO: `/v3/users/c847b76e4e6649e18dc342c5d998066b` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:38.974761 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a75994-c891-436f-9a11-d5cd0dd8f678 admin admin] RBAC: Authorizing `identity:get_user(user_id=c847b76e4e6649e18dc342c5d998066b)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:38.976601 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a75994-c891-436f-9a11-d5cd0dd8f678 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:38.978194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Mon Sep 19 10:16:38 2022] GET /identity/v3/users/c847b76e4e6649e18dc342c5d998066b => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:41.201403 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66284b7-8cd2-47c0-b2b4-d67e95dfe526 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:41.201954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66284b7-8cd2-47c0-b2b4-d67e95dfe526 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:41.201954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66284b7-8cd2-47c0-b2b4-d67e95dfe526 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:41.203869 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:16:41.209114 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a31b0-c7d2-48c5-b5a6-5074cc9cdadb None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:41.209526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a31b0-c7d2-48c5-b5a6-5074cc9cdadb None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:41.209681 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a31b0-c7d2-48c5-b5a6-5074cc9cdadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:41.216014 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332b3d3-409c-4ffb-94ab-389c10d4923b None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:41.216295 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332b3d3-409c-4ffb-94ab-389c10d4923b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:41.216567 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332b3d3-409c-4ffb-94ab-389c10d4923b None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:41.217847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:16:41.222895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8e3df-bb60-4618-b9c0-b1ad19a0cb3e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:41.223171 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8e3df-bb60-4618-b9c0-b1ad19a0cb3e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:41.223442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8e3df-bb60-4618-b9c0-b1ad19a0cb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:41.227913 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-d61a31b0-c7d2-48c5-b5a6-5074cc9cdadb None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:16:41.229092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d61a31b0-c7d2-48c5-b5a6-5074cc9cdadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:16:41.264259 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG passlib.handlers.bcrypt [None req-daa8e3df-bb60-4618-b9c0-b1ad19a0cb3e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81562) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 19 10:16:41.264638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG passlib.handlers.bcrypt [None req-daa8e3df-bb60-4618-b9c0-b1ad19a0cb3e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81562) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 19 10:16:41.310702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:16:41.314283 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-daa8e3df-bb60-4618-b9c0-b1ad19a0cb3e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:16:41.315865 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-daa8e3df-bb60-4618-b9c0-b1ad19a0cb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:16:41.418115 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:16:41.582082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a31b0-c7d2-48c5-b5a6-5074cc9cdadb None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:41.605309 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:41.689283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a31b0-c7d2-48c5-b5a6-5074cc9cdadb None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:41.689600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8e3df-bb60-4618-b9c0-b1ad19a0cb3e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:41.726450 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6ca4b-c921-47e4-b6be-84bb498ba5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:41.727029 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6ca4b-c921-47e4-b6be-84bb498ba5cc None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:41.727313 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6ca4b-c921-47e4-b6be-84bb498ba5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:41.727595 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6ca4b-c921-47e4-b6be-84bb498ba5cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:41.739611 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc6ca4b-c921-47e4-b6be-84bb498ba5cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:16:41.748401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-5fc6ca4b-c921-47e4-b6be-84bb498ba5cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:16:41.748401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:16:41.748401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:16:41.748401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:16:41.748401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:16:41.748401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:16:41.748401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:16:41.748401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:16:41.748401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:16:41.748401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:16:41.748401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:16:41.748401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:16:41.748401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:16:41.748401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:16:41.748401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:16:41.748401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:16:41.749529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:16:41.749529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:16:41.749529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:16:41.749529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:16:41.749529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:16:41.749529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:16:41.749529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:16:41.749529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:16:41.749529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:16:41.749529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:16:41.749529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:16:41.749529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:16:41.749529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:16:41.749529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:16:41.749529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:16:41.749529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:16:41.750551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:16:41.750551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:16:41.750551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:16:41.750551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:16:41.750551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:16:41.750551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:16:41.750551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:16:41.750551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:16:41.750551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:16:41.750551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:16:41.750551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:16:41.750551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:16:41.750551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:16:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:16:41.755460 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6ca4b-c921-47e4-b6be-84bb498ba5cc None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:41.794909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8e3df-bb60-4618-b9c0-b1ad19a0cb3e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:41.797255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:41.849200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164316a1-8e2a-4620-9f23-1fd341f055bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:41.849200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164316a1-8e2a-4620-9f23-1fd341f055bf None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:41.849200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164316a1-8e2a-4620-9f23-1fd341f055bf None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:41.849200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164316a1-8e2a-4620-9f23-1fd341f055bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:41.849576 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:41.858607 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6ca4b-c921-47e4-b6be-84bb498ba5cc None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:41.892828 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27596e6e-2b12-44e2-844a-1f8292e9d5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:41.893463 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27596e6e-2b12-44e2-844a-1f8292e9d5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:41.893747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27596e6e-2b12-44e2-844a-1f8292e9d5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:41.894021 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27596e6e-2b12-44e2-844a-1f8292e9d5d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:41.894688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27596e6e-2b12-44e2-844a-1f8292e9d5d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:16:41.894990 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27596e6e-2b12-44e2-844a-1f8292e9d5d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:41.896473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27596e6e-2b12-44e2-844a-1f8292e9d5d3 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:41.903956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164316a1-8e2a-4620-9f23-1fd341f055bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:41.905493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164316a1-8e2a-4620-9f23-1fd341f055bf None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:41.910274 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:16:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:16:41.915876 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27596e6e-2b12-44e2-844a-1f8292e9d5d3 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:42.010858 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 8/16] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:16:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:16:42.015000 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27596e6e-2b12-44e2-844a-1f8292e9d5d3 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:42.016935 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164316a1-8e2a-4620-9f23-1fd341f055bf None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:42.019653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:42.050369 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabe247-ef17-4c41-8f43-7ed1d0484a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:42.051005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabe247-ef17-4c41-8f43-7ed1d0484a52 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:42.051284 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabe247-ef17-4c41-8f43-7ed1d0484a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:42.051563 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabe247-ef17-4c41-8f43-7ed1d0484a52 None admin] PATH_INFO: `/v3/users/c847b76e4e6649e18dc342c5d998066b` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:42.052699 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabe247-ef17-4c41-8f43-7ed1d0484a52 None admin] RBAC: Authorizing `identity:get_user(user_id=c847b76e4e6649e18dc342c5d998066b)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:42.054211 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabe247-ef17-4c41-8f43-7ed1d0484a52 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:42.055793 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:16:41 2022] GET /identity/v3/users/c847b76e4e6649e18dc342c5d998066b => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:16:42.061139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabe247-ef17-4c41-8f43-7ed1d0484a52 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:42.075633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:42.113147 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164316a1-8e2a-4620-9f23-1fd341f055bf None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:42.146435 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c42f5b-a46d-4215-a420-8a2796c28b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:42.147278 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c42f5b-a46d-4215-a420-8a2796c28b59 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:42.147557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c42f5b-a46d-4215-a420-8a2796c28b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:42.147831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c42f5b-a46d-4215-a420-8a2796c28b59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:42.158464 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabe247-ef17-4c41-8f43-7ed1d0484a52 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:42.166193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c42f5b-a46d-4215-a420-8a2796c28b59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:42.167659 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c42f5b-a46d-4215-a420-8a2796c28b59 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:42.185463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-97c42f5b-a46d-4215-a420-8a2796c28b59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:16:42.185463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:16:42.185463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:16:42.185463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:16:42.185463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:16:42.185463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:16:42.185463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:16:42.185463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:16:42.185463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:16:42.185463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:16:42.185463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:16:42.185463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:16:42.185463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:16:42.185463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:16:42.185463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:16:42.185463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:16:42.186533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:16:42.186533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:16:42.186533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:16:42.186533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:16:42.186533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:16:42.186533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:16:42.186533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:16:42.186533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:16:42.186533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:16:42.186533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:16:42.186533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:16:42.186533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:16:42.186533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:16:42.186533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:16:42.186533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:16:42.186533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:16:42.186533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:16:42.188391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:16:42.188391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:16:42.188391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:16:42.188391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 9/18] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:16:42.192823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c42f5b-a46d-4215-a420-8a2796c28b59 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:42.195405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592978b2-aa6c-45c7-bffc-ccea0bcdf7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:42.196035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592978b2-aa6c-45c7-bffc-ccea0bcdf7c4 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:42.196318 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592978b2-aa6c-45c7-bffc-ccea0bcdf7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:42.196590 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592978b2-aa6c-45c7-bffc-ccea0bcdf7c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:42.197715 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592978b2-aa6c-45c7-bffc-ccea0bcdf7c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:42.199160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592978b2-aa6c-45c7-bffc-ccea0bcdf7c4 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:42.200693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 10/19] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:16:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:42.206836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592978b2-aa6c-45c7-bffc-ccea0bcdf7c4 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:42.248929 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:42.285032 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:42.292112 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c42f5b-a46d-4215-a420-8a2796c28b59 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:42.304691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592978b2-aa6c-45c7-bffc-ccea0bcdf7c4 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:42.326622 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4b5b0-8543-43f3-a741-b41d66614528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:42.327235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4b5b0-8543-43f3-a741-b41d66614528 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:42.327514 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4b5b0-8543-43f3-a741-b41d66614528 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:42.327789 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4b5b0-8543-43f3-a741-b41d66614528 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:42.328464 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4b5b0-8543-43f3-a741-b41d66614528 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:16:42.328769 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4b5b0-8543-43f3-a741-b41d66614528 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:42.330187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4b5b0-8543-43f3-a741-b41d66614528 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:42.339289 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51e467-08aa-4308-aea6-a8673d86a3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:42.339897 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51e467-08aa-4308-aea6-a8673d86a3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:42.340180 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51e467-08aa-4308-aea6-a8673d86a3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:42.340455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51e467-08aa-4308-aea6-a8673d86a3f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:42.341208 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b51e467-08aa-4308-aea6-a8673d86a3f9 None admin] RBAC: Adding query filter params (role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.domain.id=default, user.id=c847b76e4e6649e18dc342c5d998066b) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:16:42.341518 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b51e467-08aa-4308-aea6-a8673d86a3f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:42.342949 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b51e467-08aa-4308-aea6-a8673d86a3f9 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:42.347479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 10/20] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:16:42.350029 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 11/21] 10.222.0.42 () {56 vars in 1322 bytes} [Mon Sep 19 10:16:42 2022] GET /identity/v3/role_assignments?user.id=c847b76e4e6649e18dc342c5d998066b&scope.domain.id=default&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:16:44.694024 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da691feb-6208-49fb-87db-1fee02d1989f None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:44.694563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da691feb-6208-49fb-87db-1fee02d1989f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:44.694563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da691feb-6208-49fb-87db-1fee02d1989f None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:44.696035 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 11/22] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:16:44.702649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfe53d-d987-49ae-b508-f969ff485f16 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:44.703641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfe53d-d987-49ae-b508-f969ff485f16 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:44.703717 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfe53d-d987-49ae-b508-f969ff485f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:44.733940 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-10cfe53d-d987-49ae-b508-f969ff485f16 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:16:44.735368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-10cfe53d-d987-49ae-b508-f969ff485f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:16:44.825793 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 12/23] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:16:44.846902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6a134-e96b-4e4e-9b0f-77c3ad97e137 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:44.847505 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6a134-e96b-4e4e-9b0f-77c3ad97e137 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:44.847993 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6a134-e96b-4e4e-9b0f-77c3ad97e137 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:44.849550 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 12/24] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:16:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:16:44.854993 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fff12-cee7-4880-814a-47998f9cf463 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:44.855284 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fff12-cee7-4880-814a-47998f9cf463 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:44.855566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fff12-cee7-4880-814a-47998f9cf463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:44.881410 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-8e3fff12-cee7-4880-814a-47998f9cf463 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:16:44.882641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-8e3fff12-cee7-4880-814a-47998f9cf463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:16:44.989526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 13/25] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:16:45.416529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6a134-e96b-4e4e-9b0f-77c3ad97e137 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:45.470361 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00f7f6-bc10-4dae-a3d2-5075fcad58c1 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:45.470989 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00f7f6-bc10-4dae-a3d2-5075fcad58c1 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:45.471465 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00f7f6-bc10-4dae-a3d2-5075fcad58c1 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:45.473016 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 14/26] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:16:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:16:45.479469 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0a198-57d0-46db-9fcd-f52d40075a50 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:45.480091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0a198-57d0-46db-9fcd-f52d40075a50 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:45.480567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0a198-57d0-46db-9fcd-f52d40075a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:45.490609 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:45.507206 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-48a0a198-57d0-46db-9fcd-f52d40075a50 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:16:45.508442 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-48a0a198-57d0-46db-9fcd-f52d40075a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:16:45.602874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6a134-e96b-4e4e-9b0f-77c3ad97e137 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:45.630955 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:45.641304 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 15/27] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:16:45.642519 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0a198-57d0-46db-9fcd-f52d40075a50 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:45.653162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a82f35-955b-4e9f-ae66-384d74f7b116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:45.654075 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a82f35-955b-4e9f-ae66-384d74f7b116 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:45.654585 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a82f35-955b-4e9f-ae66-384d74f7b116 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:45.655046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a82f35-955b-4e9f-ae66-384d74f7b116 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:45.667773 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a82f35-955b-4e9f-ae66-384d74f7b116 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:16:45.676052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-14a82f35-955b-4e9f-ae66-384d74f7b116 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:16:45.676052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:16:45.676052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:16:45.676052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:16:45.676052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:16:45.676052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:16:45.676052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:16:45.676052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:16:45.676052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:16:45.676052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:16:45.676052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:16:45.676052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:16:45.676052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:16:45.676052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:16:45.676052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:16:45.676052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:16:45.676916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:16:45.676916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:16:45.676916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:16:45.676916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:16:45.676916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:16:45.676916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:16:45.676916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:16:45.676916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:16:45.676916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:16:45.676916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:16:45.676916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:16:45.676916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:16:45.676916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:16:45.676916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:16:45.676916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:16:45.676916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:16:45.677716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:16:45.677716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:16:45.677716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:16:45.677716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:16:45.677716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:16:45.677716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:16:45.677716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:16:45.677716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:16:45.677716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:16:45.677716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:16:45.677716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:16:45.677716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:16:45.680066 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 13/28] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:16:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:16:45.681431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc7d69-0c29-42cf-a7c0-2ed95625af49 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:45.687570 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc7d69-0c29-42cf-a7c0-2ed95625af49 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:45.687570 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc7d69-0c29-42cf-a7c0-2ed95625af49 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:45.687570 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 14/29] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:16:45.689207 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c639c-8ed7-414b-b789-7b1af178ba3c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:45.689698 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c639c-8ed7-414b-b789-7b1af178ba3c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:45.690188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c639c-8ed7-414b-b789-7b1af178ba3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:45.724800 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-ff5c639c-8ed7-414b-b789-7b1af178ba3c None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:16:45.732075 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ff5c639c-8ed7-414b-b789-7b1af178ba3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:16:45.847880 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0a198-57d0-46db-9fcd-f52d40075a50 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:45.884321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 15/30] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:16:45.890008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c639c-8ed7-414b-b789-7b1af178ba3c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:45.930189 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8f828-ba07-40ab-a0a5-954c0511bcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:45.932689 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8f828-ba07-40ab-a0a5-954c0511bcf6 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:45.935293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8f828-ba07-40ab-a0a5-954c0511bcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:45.936743 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8f828-ba07-40ab-a0a5-954c0511bcf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:45.942743 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:45.960983 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8f828-ba07-40ab-a0a5-954c0511bcf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:45.962913 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8f828-ba07-40ab-a0a5-954c0511bcf6 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:45.986754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-74c8f828-ba07-40ab-a0a5-954c0511bcf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:16:45.986754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:16:45.986754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:16:45.986754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:16:45.986754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:16:45.986754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:16:45.986754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:16:45.986754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:16:45.986754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:16:45.986754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:16:45.986754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:16:45.986754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:16:45.986754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:16:45.986754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:16:45.986754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:16:45.986754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:16:45.987711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:16:45.987711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:16:45.987711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:16:45.987711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:16:45.987711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:16:45.987711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:16:45.987711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:16:45.987711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:16:45.987711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:16:45.987711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:16:45.987711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:16:45.987711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:16:45.987711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:16:45.987711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:16:45.987711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:16:45.987711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:16:45.987711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:16:45.988552 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:16:45.988552 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:16:45.988552 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:16:45.994862 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 16/31] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:16:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:16:45.996631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8f828-ba07-40ab-a0a5-954c0511bcf6 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:46.089954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c639c-8ed7-414b-b789-7b1af178ba3c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:46.102836 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:46.151679 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8f828-ba07-40ab-a0a5-954c0511bcf6 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:46.158889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f875262-da24-4ab9-a128-51de3d490200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:46.159783 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f875262-da24-4ab9-a128-51de3d490200 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:46.160263 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f875262-da24-4ab9-a128-51de3d490200 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:46.160736 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f875262-da24-4ab9-a128-51de3d490200 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:46.161669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f875262-da24-4ab9-a128-51de3d490200 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:16:46.162176 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f875262-da24-4ab9-a128-51de3d490200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:46.163757 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f875262-da24-4ab9-a128-51de3d490200 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:46.188946 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 16/32] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:16:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:46.190789 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f875262-da24-4ab9-a128-51de3d490200 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:46.214309 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5966a9-8e61-4b14-a696-ca71f389a4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:46.215258 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5966a9-8e61-4b14-a696-ca71f389a4a4 None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:46.215681 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5966a9-8e61-4b14-a696-ca71f389a4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:46.216097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5966a9-8e61-4b14-a696-ca71f389a4a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:46.217031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5966a9-8e61-4b14-a696-ca71f389a4a4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:46.219148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5966a9-8e61-4b14-a696-ca71f389a4a4 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:46.304158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f875262-da24-4ab9-a128-51de3d490200 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:46.343041 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a66b1-080d-4e49-a917-96ffae603474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:46.344689 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a66b1-080d-4e49-a917-96ffae603474 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:46.345232 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a66b1-080d-4e49-a917-96ffae603474 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:46.345726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a66b1-080d-4e49-a917-96ffae603474 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:46.346761 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a66b1-080d-4e49-a917-96ffae603474 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:16:46.347268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a66b1-080d-4e49-a917-96ffae603474 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:46.348911 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a66b1-080d-4e49-a917-96ffae603474 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:46.362034 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 17/33] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:46.363295 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a66b1-080d-4e49-a917-96ffae603474 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:46.403989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:46.465187 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 17/34] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Sep 19 10:16:45 2022] POST /identity/v3/roles => generated 219 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:16:46.465187 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5966a9-8e61-4b14-a696-ca71f389a4a4 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:46.511676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a66b1-080d-4e49-a917-96ffae603474 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:46.540029 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:46.560424 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a942f-2083-4d64-b7cf-816b1f993188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:46.561086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a942f-2083-4d64-b7cf-816b1f993188 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:46.561384 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a942f-2083-4d64-b7cf-816b1f993188 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:46.561664 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a942f-2083-4d64-b7cf-816b1f993188 None admin] PATH_INFO: `/v3/users/c847b76e4e6649e18dc342c5d998066b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:46.563239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a942f-2083-4d64-b7cf-816b1f993188 None admin] RBAC: Authorizing `identity:get_user(user_id=c847b76e4e6649e18dc342c5d998066b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:46.564728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a942f-2083-4d64-b7cf-816b1f993188 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:46.571300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 18/35] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:16:46 2022] GET /identity/v3/users/c847b76e4e6649e18dc342c5d998066b => generated 247 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:46.571300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a942f-2083-4d64-b7cf-816b1f993188 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:46.607173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5966a9-8e61-4b14-a696-ca71f389a4a4 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:46.651193 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3a3ce-e5a8-4632-a222-fcf844dfe144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:46.651817 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3a3ce-e5a8-4632-a222-fcf844dfe144 None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:46.652098 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3a3ce-e5a8-4632-a222-fcf844dfe144 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:46.652375 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3a3ce-e5a8-4632-a222-fcf844dfe144 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:46.653199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3a3ce-e5a8-4632-a222-fcf844dfe144 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:46.654783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3a3ce-e5a8-4632-a222-fcf844dfe144 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:46.682819 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:46.688497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a942f-2083-4d64-b7cf-816b1f993188 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:46.720024 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 18/36] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Sep 19 10:16:46 2022] POST /identity/v3/roles => generated 225 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:16:46.721449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5fe6c-bbd8-4fa9-9404-04d6bf71d2a7 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:46.721748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5fe6c-bbd8-4fa9-9404-04d6bf71d2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:46.722028 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5fe6c-bbd8-4fa9-9404-04d6bf71d2a7 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:46.728081 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 19/37] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:16:46.728081 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c59cac-4b75-440b-ae38-b3324d65a64b None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:46.728081 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c59cac-4b75-440b-ae38-b3324d65a64b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:46.728081 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c59cac-4b75-440b-ae38-b3324d65a64b None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:46.728081 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 20/38] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:16:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:16:46.729618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e01736-1b1a-47b0-89f5-db41f73bc0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:46.730271 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e01736-1b1a-47b0-89f5-db41f73bc0f1 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:46.730756 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e01736-1b1a-47b0-89f5-db41f73bc0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:46.731095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e01736-1b1a-47b0-89f5-db41f73bc0f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:46.731307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c59cac-4b75-440b-ae38-b3324d65a64b None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:46.731846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e01736-1b1a-47b0-89f5-db41f73bc0f1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:46.733523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e01736-1b1a-47b0-89f5-db41f73bc0f1 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:46.848723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c59cac-4b75-440b-ae38-b3324d65a64b None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:46.857902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:46.889628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aacde51-86c4-4256-88b0-b446e4b749e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:46.891235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacde51-86c4-4256-88b0-b446e4b749e1 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:46.891235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacde51-86c4-4256-88b0-b446e4b749e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:46.891381 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacde51-86c4-4256-88b0-b446e4b749e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:46.892551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aacde51-86c4-4256-88b0-b446e4b749e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:46.894099 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aacde51-86c4-4256-88b0-b446e4b749e1 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:46.895708 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 21/39] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:16:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:46.896819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a0cc3-64b1-41f5-bfb0-40070827e920 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:46.897048 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a0cc3-64b1-41f5-bfb0-40070827e920 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:46.897347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a0cc3-64b1-41f5-bfb0-40070827e920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:46.917929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-2c7a0cc3-64b1-41f5-bfb0-40070827e920 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:16:46.919462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-2c7a0cc3-64b1-41f5-bfb0-40070827e920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:16:46.944555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 19/40] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 19 10:16:46 2022] POST /identity/v3/projects => generated 301 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:16:46.944555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6282a7-651c-44ee-8362-d2adcc5d2ee5 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:46.944555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6282a7-651c-44ee-8362-d2adcc5d2ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:46.944555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6282a7-651c-44ee-8362-d2adcc5d2ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:46.968394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-ce6282a7-651c-44ee-8362-d2adcc5d2ee5 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:16:46.971853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ce6282a7-651c-44ee-8362-d2adcc5d2ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:16:46.999305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 22/41] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:16:47.000227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a0cc3-64b1-41f5-bfb0-40070827e920 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:47.054923 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 20/42] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:16:47.229820 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a0cc3-64b1-41f5-bfb0-40070827e920 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:47.232752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:47.263816 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8f25c-bdea-4939-aab9-dbbfd95b35f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:47.264484 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8f25c-bdea-4939-aab9-dbbfd95b35f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:47.264770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8f25c-bdea-4939-aab9-dbbfd95b35f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:47.265052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8f25c-bdea-4939-aab9-dbbfd95b35f0 None admin] PATH_INFO: `/v3/domains/default/users/c847b76e4e6649e18dc342c5d998066b/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:47.267537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8f25c-bdea-4939-aab9-dbbfd95b35f0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c847b76e4e6649e18dc342c5d998066b, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:47.269024 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8f25c-bdea-4939-aab9-dbbfd95b35f0 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:47.281659 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6282a7-651c-44ee-8362-d2adcc5d2ee5 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:47.335078 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 23/43] 10.222.0.42 () {58 vars in 1361 bytes} [Mon Sep 19 10:16:46 2022] PUT /identity/v3/domains/default/users/c847b76e4e6649e18dc342c5d998066b/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:16:47.335978 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8f25c-bdea-4939-aab9-dbbfd95b35f0 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:47.368387 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:47.393790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6282a7-651c-44ee-8362-d2adcc5d2ee5 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:47.435898 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad01488-3ce8-4ce7-ba4b-fd24225bd824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:47.436567 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad01488-3ce8-4ce7-ba4b-fd24225bd824 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:47.436870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad01488-3ce8-4ce7-ba4b-fd24225bd824 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:47.437934 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad01488-3ce8-4ce7-ba4b-fd24225bd824 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:47.439821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad01488-3ce8-4ce7-ba4b-fd24225bd824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:47.441600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad01488-3ce8-4ce7-ba4b-fd24225bd824 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:47.443413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 21/44] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:16:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:16:47.449024 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad01488-3ce8-4ce7-ba4b-fd24225bd824 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:47.450516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8f25c-bdea-4939-aab9-dbbfd95b35f0 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:47.490587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ebd98-6699-4ce2-ae0b-e7f7973a083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:47.491139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ebd98-6699-4ce2-ae0b-e7f7973a083a None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:47.491427 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ebd98-6699-4ce2-ae0b-e7f7973a083a None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:47.491709 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ebd98-6699-4ce2-ae0b-e7f7973a083a None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:47.492473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ebd98-6699-4ce2-ae0b-e7f7973a083a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:47.493964 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ebd98-6699-4ce2-ae0b-e7f7973a083a None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:47.559213 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad01488-3ce8-4ce7-ba4b-fd24225bd824 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:47.560211 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 24/45] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Sep 19 10:16:47 2022] POST /identity/v3/roles => generated 223 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:16:47.597517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20581d-7822-41c7-9a0c-8a9134187bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:47.598154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20581d-7822-41c7-9a0c-8a9134187bee None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:47.598441 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20581d-7822-41c7-9a0c-8a9134187bee None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:47.598719 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20581d-7822-41c7-9a0c-8a9134187bee None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:47.599393 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20581d-7822-41c7-9a0c-8a9134187bee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:47.600938 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20581d-7822-41c7-9a0c-8a9134187bee None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:47.662670 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:47.669687 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 22/46] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 19 10:16:47 2022] POST /identity/v3/projects => generated 312 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:16:49.477328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65a53e-37e3-4253-829d-63f4fe220fd9 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:49.477849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65a53e-37e3-4253-829d-63f4fe220fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:49.477849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65a53e-37e3-4253-829d-63f4fe220fd9 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:49.479204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 25/47] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:16:49.484565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbca1ee-d137-4660-9340-ad54aeb33cff None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:49.484981 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbca1ee-d137-4660-9340-ad54aeb33cff None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:49.485146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbca1ee-d137-4660-9340-ad54aeb33cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:49.503172 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-bcbca1ee-d137-4660-9340-ad54aeb33cff None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:16:49.504357 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bcbca1ee-d137-4660-9340-ad54aeb33cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:16:49.569284 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 23/48] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:16:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:16:49.823406 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee3ba1-3322-4981-a568-43ad3387b215 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:49.824104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee3ba1-3322-4981-a568-43ad3387b215 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:49.824104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee3ba1-3322-4981-a568-43ad3387b215 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:49.825239 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 26/49] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:16:49.831229 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061fc47-3a49-4e08-87f3-26fdaeaa3ccb None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:49.831589 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061fc47-3a49-4e08-87f3-26fdaeaa3ccb None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:49.831936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061fc47-3a49-4e08-87f3-26fdaeaa3ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:49.837620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee3ba1-3322-4981-a568-43ad3387b215 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:49.850471 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7061fc47-3a49-4e08-87f3-26fdaeaa3ccb None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:16:49.851678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7061fc47-3a49-4e08-87f3-26fdaeaa3ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:16:49.863095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:49.917821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 24/50] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:16:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:16:49.940044 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee3ba1-3322-4981-a568-43ad3387b215 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:49.974891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ce0ee-3b26-408a-b991-41272533481a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:49.975521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ce0ee-3b26-408a-b991-41272533481a None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:49.975807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ce0ee-3b26-408a-b991-41272533481a None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:49.976087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ce0ee-3b26-408a-b991-41272533481a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:49.985980 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ce0ee-3b26-408a-b991-41272533481a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:16:49.991282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-ea6ce0ee-3b26-408a-b991-41272533481a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:16:49.991282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:16:49.991282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:16:49.991282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:16:49.991282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:16:49.991282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:16:49.991282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:16:49.991282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:16:49.991282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:16:49.991282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:16:49.991282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:16:49.991282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:16:49.991282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:16:49.991282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:16:49.991282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:16:49.991282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:16:49.992104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:16:49.992104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:16:49.992104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:16:49.992104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:16:49.992104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:16:49.992104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:16:49.992104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:16:49.992104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:16:49.992104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:16:49.992104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:16:49.992104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:16:49.992104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:16:49.992104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:16:49.992104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:16:49.992104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:16:49.992104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:16:49.992849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:16:49.992849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:16:49.992849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:16:49.992849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:16:49.992849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:16:49.992849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:16:49.992849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:16:49.992849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:16:49.992849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:16:49.992849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:16:49.992849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:16:49.992849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:16:49.992849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 27/51] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:16:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:16:49.998301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061fc47-3a49-4e08-87f3-26fdaeaa3ccb None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:50.055079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:50.187635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ce0ee-3b26-408a-b991-41272533481a None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:50.198973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061fc47-3a49-4e08-87f3-26fdaeaa3ccb None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:50.233556 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f75e22-5358-4952-aee0-1032d9bb947b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:50.234198 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f75e22-5358-4952-aee0-1032d9bb947b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:50.234479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f75e22-5358-4952-aee0-1032d9bb947b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:50.234755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f75e22-5358-4952-aee0-1032d9bb947b None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:50.235423 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f75e22-5358-4952-aee0-1032d9bb947b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:16:50.235732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f75e22-5358-4952-aee0-1032d9bb947b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:50.237140 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f75e22-5358-4952-aee0-1032d9bb947b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:50.244984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 25/52] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:16:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:50.251126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f75e22-5358-4952-aee0-1032d9bb947b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:50.276264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:50.292355 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ce0ee-3b26-408a-b991-41272533481a None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:50.326724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbee85-8e87-43e7-92a2-dd6a39119e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:50.327340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbee85-8e87-43e7-92a2-dd6a39119e9c None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:50.327662 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbee85-8e87-43e7-92a2-dd6a39119e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:50.328561 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbee85-8e87-43e7-92a2-dd6a39119e9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:50.329666 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddbee85-8e87-43e7-92a2-dd6a39119e9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:50.331107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddbee85-8e87-43e7-92a2-dd6a39119e9c None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:50.332643 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 28/53] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:16:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:16:50.338066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbee85-8e87-43e7-92a2-dd6a39119e9c None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:50.347566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f75e22-5358-4952-aee0-1032d9bb947b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:50.366699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:50.380838 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b645dcd-2e39-42f0-8c0c-ac45c93e9cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:50.381498 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b645dcd-2e39-42f0-8c0c-ac45c93e9cee None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:50.381784 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b645dcd-2e39-42f0-8c0c-ac45c93e9cee None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:50.382062 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b645dcd-2e39-42f0-8c0c-ac45c93e9cee None admin] PATH_INFO: `/v3/users/c847b76e4e6649e18dc342c5d998066b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:50.383202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b645dcd-2e39-42f0-8c0c-ac45c93e9cee None admin] RBAC: Authorizing `identity:get_user(user_id=c847b76e4e6649e18dc342c5d998066b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:50.384633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b645dcd-2e39-42f0-8c0c-ac45c93e9cee None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:50.386217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 26/54] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:16:50 2022] GET /identity/v3/users/c847b76e4e6649e18dc342c5d998066b => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:16:50.392135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b645dcd-2e39-42f0-8c0c-ac45c93e9cee None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:50.439760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbee85-8e87-43e7-92a2-dd6a39119e9c None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:50.474119 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e6ec8-5957-460f-a52b-0757cbbfe72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:50.474734 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e6ec8-5957-460f-a52b-0757cbbfe72c None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:50.475017 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e6ec8-5957-460f-a52b-0757cbbfe72c None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:50.475294 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e6ec8-5957-460f-a52b-0757cbbfe72c None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:50.475947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e6ec8-5957-460f-a52b-0757cbbfe72c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:50.477476 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e6ec8-5957-460f-a52b-0757cbbfe72c None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:50.491218 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b645dcd-2e39-42f0-8c0c-ac45c93e9cee None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:50.524406 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9376848-ce04-4c61-b9cf-63ab1ae6c79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:50.525028 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9376848-ce04-4c61-b9cf-63ab1ae6c79c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:50.525343 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9376848-ce04-4c61-b9cf-63ab1ae6c79c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:50.525859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9376848-ce04-4c61-b9cf-63ab1ae6c79c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:50.526962 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9376848-ce04-4c61-b9cf-63ab1ae6c79c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:50.528571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9376848-ce04-4c61-b9cf-63ab1ae6c79c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:50.530149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 27/55] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:16:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:16:50.535898 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9376848-ce04-4c61-b9cf-63ab1ae6c79c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:50.589754 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:50.593559 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 29/56] 10.222.0.42 () {60 vars in 1140 bytes} [Mon Sep 19 10:16:50 2022] POST /identity/v3/projects => generated 298 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:16:50.640569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9376848-ce04-4c61-b9cf-63ab1ae6c79c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:50.677285 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0767b-8a2a-4528-895f-b9049ca3b83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:50.677908 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0767b-8a2a-4528-895f-b9049ca3b83c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:50.678190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0767b-8a2a-4528-895f-b9049ca3b83c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:50.678550 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0767b-8a2a-4528-895f-b9049ca3b83c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:50.679296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b0767b-8a2a-4528-895f-b9049ca3b83c None admin] RBAC: Adding query filter params (role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.domain.id=default, user.id=c847b76e4e6649e18dc342c5d998066b) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:16:50.679608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b0767b-8a2a-4528-895f-b9049ca3b83c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:50.681208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b0767b-8a2a-4528-895f-b9049ca3b83c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:50.691406 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 28/57] 10.222.0.42 () {56 vars in 1322 bytes} [Mon Sep 19 10:16:50 2022] GET /identity/v3/role_assignments?user.id=c847b76e4e6649e18dc342c5d998066b&scope.domain.id=default&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:16:52.745220 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15e3bc-c38c-41e3-848b-5b7a20a2fe6c None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:52.745733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15e3bc-c38c-41e3-848b-5b7a20a2fe6c None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:52.745836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15e3bc-c38c-41e3-848b-5b7a20a2fe6c None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:52.747094 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 30/58] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:16:52.752919 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a751622-0da3-44fb-a4c6-8ab5fec2f416 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:52.753340 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a751622-0da3-44fb-a4c6-8ab5fec2f416 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:52.753506 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a751622-0da3-44fb-a4c6-8ab5fec2f416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:52.771193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7a751622-0da3-44fb-a4c6-8ab5fec2f416 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:16:52.772376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7a751622-0da3-44fb-a4c6-8ab5fec2f416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:16:52.836611 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 29/59] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:16:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:16:53.108095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15e3bc-c38c-41e3-848b-5b7a20a2fe6c None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:53.112459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:53.207371 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15e3bc-c38c-41e3-848b-5b7a20a2fe6c None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:53.240193 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19dc836-2c45-4fe4-b113-029639c23040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:53.240815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dc836-2c45-4fe4-b113-029639c23040 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:53.241120 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dc836-2c45-4fe4-b113-029639c23040 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:53.241402 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dc836-2c45-4fe4-b113-029639c23040 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:53.242521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19dc836-2c45-4fe4-b113-029639c23040 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:53.243962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19dc836-2c45-4fe4-b113-029639c23040 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:53.245576 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 31/60] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:16:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:53.251075 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a751622-0da3-44fb-a4c6-8ab5fec2f416 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:53.298819 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:53.347628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a751622-0da3-44fb-a4c6-8ab5fec2f416 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:53.380450 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf154f21-3770-49fe-bdf6-fc5683aa24f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:53.381001 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf154f21-3770-49fe-bdf6-fc5683aa24f8 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:53.381316 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf154f21-3770-49fe-bdf6-fc5683aa24f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:53.381597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf154f21-3770-49fe-bdf6-fc5683aa24f8 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:53.382259 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf154f21-3770-49fe-bdf6-fc5683aa24f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:53.383694 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf154f21-3770-49fe-bdf6-fc5683aa24f8 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:53.482590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 30/61] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 19 10:16:53 2022] POST /identity/v3/users => generated 275 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:16:56.742057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3ca6d-dfc5-4bc7-8f25-16bb84392a87 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:56.743061 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3ca6d-dfc5-4bc7-8f25-16bb84392a87 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:56.743582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3ca6d-dfc5-4bc7-8f25-16bb84392a87 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:56.745461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 32/62] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:16:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:16:56.751329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3563bd-c1e3-4c32-83fb-368e2a1ccc1e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:56.752193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3563bd-c1e3-4c32-83fb-368e2a1ccc1e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:56.752677 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3563bd-c1e3-4c32-83fb-368e2a1ccc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:56.781322 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6c3563bd-c1e3-4c32-83fb-368e2a1ccc1e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:16:56.782914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6c3563bd-c1e3-4c32-83fb-368e2a1ccc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:16:56.790399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caac1b-20f2-4966-9ade-bd93510adcc5 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:56.790973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caac1b-20f2-4966-9ade-bd93510adcc5 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:56.791447 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caac1b-20f2-4966-9ade-bd93510adcc5 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:56.793305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 33/63] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:16:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:16:56.818721 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abe5d1-8364-42b1-9de2-3aaaabd658a5 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:56.824514 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abe5d1-8364-42b1-9de2-3aaaabd658a5 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:56.825005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abe5d1-8364-42b1-9de2-3aaaabd658a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:56.845518 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-e2abe5d1-8364-42b1-9de2-3aaaabd658a5 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:16:56.847087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e2abe5d1-8364-42b1-9de2-3aaaabd658a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:16:56.919171 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 31/64] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:16:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:16:56.928394 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 34/65] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:16:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:16:57.171744 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e15f9d-b383-4750-80df-1d45c89b2065 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:57.172429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e15f9d-b383-4750-80df-1d45c89b2065 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:57.172945 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e15f9d-b383-4750-80df-1d45c89b2065 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:57.174776 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 32/66] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:16:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:16:57.179704 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f5f44-f6d1-4233-812d-352f1c8b53a2 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:57.180222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f5f44-f6d1-4233-812d-352f1c8b53a2 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:57.180718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f5f44-f6d1-4233-812d-352f1c8b53a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:57.193085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:57.205964 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-ca3f5f44-f6d1-4233-812d-352f1c8b53a2 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:16:57.207407 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-ca3f5f44-f6d1-4233-812d-352f1c8b53a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:16:57.248195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e15f9d-b383-4750-80df-1d45c89b2065 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:57.298456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 35/67] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:16:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:16:57.300423 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f5f44-f6d1-4233-812d-352f1c8b53a2 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:57.311954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:57.438097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e15f9d-b383-4750-80df-1d45c89b2065 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:57.472509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f5f44-f6d1-4233-812d-352f1c8b53a2 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:57.484639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452e501-b7f6-4622-80e1-9a98d534ad8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:57.485388 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452e501-b7f6-4622-80e1-9a98d534ad8b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:57.485670 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452e501-b7f6-4622-80e1-9a98d534ad8b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:57.485945 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452e501-b7f6-4622-80e1-9a98d534ad8b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:57.497602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2452e501-b7f6-4622-80e1-9a98d534ad8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:16:57.503148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-2452e501-b7f6-4622-80e1-9a98d534ad8b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:16:57.503148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:16:57.503148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:16:57.503148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:16:57.503148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:16:57.503148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:16:57.503148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:16:57.503148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:16:57.503148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:16:57.503148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:16:57.503148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:16:57.503148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:16:57.503148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:16:57.503148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:16:57.503148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:16:57.503148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:16:57.504020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:16:57.504020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:16:57.504020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:16:57.504020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:16:57.504020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:16:57.504020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:16:57.504020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:16:57.504020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:16:57.504020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:16:57.504020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:16:57.504020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:16:57.504020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:16:57.504020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:16:57.504020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:16:57.504020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:16:57.504020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:16:57.504791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:16:57.504791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:16:57.504791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:16:57.504791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:16:57.504791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:16:57.504791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:16:57.504791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:16:57.504791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:16:57.504791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:16:57.504791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:16:57.504791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:16:57.504791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:16:57.504791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 33/68] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:16:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:16:57.510019 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:57.510751 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef0c8e-dcce-4eef-9cd3-0fdc66b28b4a None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:57.511049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef0c8e-dcce-4eef-9cd3-0fdc66b28b4a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:57.511339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef0c8e-dcce-4eef-9cd3-0fdc66b28b4a None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:57.512747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 34/69] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:16:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:16:57.517603 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef0c8e-dcce-4eef-9cd3-0fdc66b28b4a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:57.521716 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a816f-5e67-498b-ab6a-f640591859ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:57.521716 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a816f-5e67-498b-ab6a-f640591859ab None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:57.521716 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a816f-5e67-498b-ab6a-f640591859ab None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:57.521716 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a816f-5e67-498b-ab6a-f640591859ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:57.533553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a816f-5e67-498b-ab6a-f640591859ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:16:57.539168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-d08a816f-5e67-498b-ab6a-f640591859ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:16:57.539168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:16:57.539168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:16:57.539168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:16:57.539168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:16:57.539168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:16:57.539168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:16:57.539168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:16:57.539168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:16:57.539168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:16:57.539168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:16:57.539168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:16:57.539168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:16:57.539168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:16:57.539168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:16:57.539168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:16:57.540131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:16:57.540131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:16:57.540131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:16:57.540131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:16:57.540131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:16:57.540131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:16:57.540131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:16:57.540131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:16:57.540131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:16:57.540131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:16:57.540131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:16:57.540131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:16:57.540131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:16:57.540131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:16:57.540131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:16:57.540131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:16:57.540899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:16:57.540899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:16:57.540899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:16:57.540899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:16:57.540899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:16:57.540899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:16:57.540899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:16:57.540899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:16:57.540899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:16:57.540899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:16:57.540899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:16:57.540899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:16:57.543502 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 36/70] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:16:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:16:57.544974 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55927326-fc62-4f85-80e0-5b3c77911b04 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:57.545501 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55927326-fc62-4f85-80e0-5b3c77911b04 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:57.545992 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55927326-fc62-4f85-80e0-5b3c77911b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:57.565996 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-55927326-fc62-4f85-80e0-5b3c77911b04 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:16:57.567536 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-55927326-fc62-4f85-80e0-5b3c77911b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:16:57.622994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:57.631034 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef0c8e-dcce-4eef-9cd3-0fdc66b28b4a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:57.635306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 37/71] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:16:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:16:57.645222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55927326-fc62-4f85-80e0-5b3c77911b04 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:57.665988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f97dc1-bb26-465b-9c5e-1c91e6d03166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:57.666689 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f97dc1-bb26-465b-9c5e-1c91e6d03166 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:57.667041 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f97dc1-bb26-465b-9c5e-1c91e6d03166 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:57.667439 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f97dc1-bb26-465b-9c5e-1c91e6d03166 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:57.668157 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f97dc1-bb26-465b-9c5e-1c91e6d03166 None admin] RBAC: Adding query filter params (name=member) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:16:57.668467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f97dc1-bb26-465b-9c5e-1c91e6d03166 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:57.669897 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f97dc1-bb26-465b-9c5e-1c91e6d03166 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:57.678332 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 35/72] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:16:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:57.679400 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5615c32-5e4f-4496-bb80-aaf8b517275b None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:57.679691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5615c32-5e4f-4496-bb80-aaf8b517275b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:57.679963 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5615c32-5e4f-4496-bb80-aaf8b517275b None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:57.681182 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 36/73] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:16:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:16:57.681879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5615c32-5e4f-4496-bb80-aaf8b517275b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:57.747890 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55927326-fc62-4f85-80e0-5b3c77911b04 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:57.771346 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:57.783784 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7439fe6-1d5f-4e96-a1a5-675da5d3d8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:57.784734 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7439fe6-1d5f-4e96-a1a5-675da5d3d8dc None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:57.785073 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7439fe6-1d5f-4e96-a1a5-675da5d3d8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:57.785356 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7439fe6-1d5f-4e96-a1a5-675da5d3d8dc None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:57.786036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7439fe6-1d5f-4e96-a1a5-675da5d3d8dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:16:57.786345 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7439fe6-1d5f-4e96-a1a5-675da5d3d8dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:57.787728 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7439fe6-1d5f-4e96-a1a5-675da5d3d8dc None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:57.789465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5615c32-5e4f-4496-bb80-aaf8b517275b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:57.801025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 38/74] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:16:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:57.801025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7439fe6-1d5f-4e96-a1a5-675da5d3d8dc None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:57.834947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daab34b-d0c0-4b7a-961a-559f19dc2810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:57.835586 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daab34b-d0c0-4b7a-961a-559f19dc2810 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:57.835866 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daab34b-d0c0-4b7a-961a-559f19dc2810 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:57.836142 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daab34b-d0c0-4b7a-961a-559f19dc2810 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:57.846500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daab34b-d0c0-4b7a-961a-559f19dc2810 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:16:57.851376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-3daab34b-d0c0-4b7a-961a-559f19dc2810 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:16:57.851376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:16:57.851376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:16:57.851376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:16:57.851376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:16:57.851376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:16:57.851376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:16:57.851376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:16:57.851376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:16:57.851376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:16:57.851376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:16:57.851376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:16:57.851376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:16:57.851376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:16:57.851376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:16:57.851376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:16:57.852281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:16:57.852281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:16:57.852281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:16:57.852281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:16:57.852281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:16:57.852281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:16:57.852281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:16:57.852281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:16:57.852281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:16:57.852281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:16:57.852281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:16:57.852281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:16:57.852281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:16:57.852281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:16:57.852281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:16:57.852281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:16:57.853662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:16:57.853662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:16:57.853662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:16:57.853662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:16:57.853662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:16:57.853662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:16:57.853662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:16:57.853662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:16:57.853662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:16:57.853662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:16:57.853662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:16:57.853662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:16:57.853662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 37/75] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:16:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:16:57.857773 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553bca6-eb4f-4ad3-95d8-8a465071f363 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:57.858059 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553bca6-eb4f-4ad3-95d8-8a465071f363 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:57.858335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553bca6-eb4f-4ad3-95d8-8a465071f363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:57.876834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c553bca6-eb4f-4ad3-95d8-8a465071f363 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:16:57.878063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c553bca6-eb4f-4ad3-95d8-8a465071f363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:16:57.892431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:57.905998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7439fe6-1d5f-4e96-a1a5-675da5d3d8dc None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:57.939172 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a2bde-8a76-4530-b4f4-f5ea40c77cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:57.940014 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a2bde-8a76-4530-b4f4-f5ea40c77cca None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:57.940301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a2bde-8a76-4530-b4f4-f5ea40c77cca None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:57.940572 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a2bde-8a76-4530-b4f4-f5ea40c77cca None admin] PATH_INFO: `/v3/users/fda9ca2d3fb54751b2eb0636906d0c8c` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:57.948672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 38/76] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:16:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:16:57.949568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c553bca6-eb4f-4ad3-95d8-8a465071f363 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:57.957097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a2bde-8a76-4530-b4f4-f5ea40c77cca None admin] RBAC: Authorizing `identity:get_user(user_id=fda9ca2d3fb54751b2eb0636906d0c8c)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:57.958757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a2bde-8a76-4530-b4f4-f5ea40c77cca None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:57.964710 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 39/77] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:16:57 2022] GET /identity/v3/users/fda9ca2d3fb54751b2eb0636906d0c8c => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:57.964710 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a2bde-8a76-4530-b4f4-f5ea40c77cca None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:58.006518 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:58.058919 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c553bca6-eb4f-4ad3-95d8-8a465071f363 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:58.065494 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a2bde-8a76-4530-b4f4-f5ea40c77cca None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:58.098411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac7143-94cf-43f8-8ed4-898a88aaea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:58.099049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac7143-94cf-43f8-8ed4-898a88aaea60 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:58.099432 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac7143-94cf-43f8-8ed4-898a88aaea60 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:58.099816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac7143-94cf-43f8-8ed4-898a88aaea60 None admin] PATH_INFO: `/v3/users/c847b76e4e6649e18dc342c5d998066b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:58.100624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9097097-b341-4e8a-b8d3-356f68caa76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:58.100951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac7143-94cf-43f8-8ed4-898a88aaea60 None admin] RBAC: Authorizing `identity:get_user(user_id=c847b76e4e6649e18dc342c5d998066b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:58.101265 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9097097-b341-4e8a-b8d3-356f68caa76a None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:58.101548 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9097097-b341-4e8a-b8d3-356f68caa76a None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:58.101820 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9097097-b341-4e8a-b8d3-356f68caa76a None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:58.102434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac7143-94cf-43f8-8ed4-898a88aaea60 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:58.102533 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9097097-b341-4e8a-b8d3-356f68caa76a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:16:58.102764 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9097097-b341-4e8a-b8d3-356f68caa76a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:58.108742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 39/78] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:16:57 2022] GET /identity/v3/users/c847b76e4e6649e18dc342c5d998066b => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:58.108858 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9097097-b341-4e8a-b8d3-356f68caa76a None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:58.108931 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac7143-94cf-43f8-8ed4-898a88aaea60 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:58.115961 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 40/79] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:16:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:58.121178 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9097097-b341-4e8a-b8d3-356f68caa76a None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:58.127473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:58.214412 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac7143-94cf-43f8-8ed4-898a88aaea60 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:58.223878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9097097-b341-4e8a-b8d3-356f68caa76a None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:58.226152 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:58.248636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbb4a3-7eec-4a65-9b3a-05dfeacf615f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:58.249285 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbb4a3-7eec-4a65-9b3a-05dfeacf615f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:58.249570 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbb4a3-7eec-4a65-9b3a-05dfeacf615f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:58.249845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbb4a3-7eec-4a65-9b3a-05dfeacf615f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:58.251005 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbb4a3-7eec-4a65-9b3a-05dfeacf615f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:58.252435 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbb4a3-7eec-4a65-9b3a-05dfeacf615f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:58.257213 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 40/80] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:16:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:58.257213 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbb4a3-7eec-4a65-9b3a-05dfeacf615f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:58.261198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaf7b5-113d-4282-a85c-b10a3e91eb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:58.261814 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaf7b5-113d-4282-a85c-b10a3e91eb95 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:58.262099 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaf7b5-113d-4282-a85c-b10a3e91eb95 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:58.262378 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaf7b5-113d-4282-a85c-b10a3e91eb95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:58.327981 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feaf7b5-113d-4282-a85c-b10a3e91eb95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:58.330173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feaf7b5-113d-4282-a85c-b10a3e91eb95 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:58.336522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:58.351541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbb4a3-7eec-4a65-9b3a-05dfeacf615f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:58.385037 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12520ab-88e5-42c6-8a27-4185baf0ba34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:58.385856 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12520ab-88e5-42c6-8a27-4185baf0ba34 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:58.386140 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12520ab-88e5-42c6-8a27-4185baf0ba34 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:58.386421 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12520ab-88e5-42c6-8a27-4185baf0ba34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:58.425790 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 41/81] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:16:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:16:58.426436 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaf7b5-113d-4282-a85c-b10a3e91eb95 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:58.429395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:58.459039 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12520ab-88e5-42c6-8a27-4185baf0ba34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:58.460528 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12520ab-88e5-42c6-8a27-4185baf0ba34 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:58.522095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaf7b5-113d-4282-a85c-b10a3e91eb95 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:58.539767 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:58.554305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da919403-bb42-431a-bcb9-f0da88a019ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:58.554932 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da919403-bb42-431a-bcb9-f0da88a019ed None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:58.555211 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da919403-bb42-431a-bcb9-f0da88a019ed None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:58.555485 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da919403-bb42-431a-bcb9-f0da88a019ed None admin] PATH_INFO: `/v3/users/fda9ca2d3fb54751b2eb0636906d0c8c` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:58.556118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 41/82] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:16:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:16:58.556600 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da919403-bb42-431a-bcb9-f0da88a019ed None admin] RBAC: Authorizing `identity:get_user(user_id=fda9ca2d3fb54751b2eb0636906d0c8c)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:58.556934 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12520ab-88e5-42c6-8a27-4185baf0ba34 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:58.558213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da919403-bb42-431a-bcb9-f0da88a019ed None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:58.559777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 42/83] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:16:58 2022] GET /identity/v3/users/fda9ca2d3fb54751b2eb0636906d0c8c => generated 275 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:58.560496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da919403-bb42-431a-bcb9-f0da88a019ed None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:58.630974 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:58.657218 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12520ab-88e5-42c6-8a27-4185baf0ba34 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:58.659746 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da919403-bb42-431a-bcb9-f0da88a019ed None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:58.690179 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c2778-d113-4110-9285-c2aa54c5d03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:58.690809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c2778-d113-4110-9285-c2aa54c5d03f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:58.691097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c2778-d113-4110-9285-c2aa54c5d03f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:58.691369 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c2778-d113-4110-9285-c2aa54c5d03f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:58.693320 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433d9eb-3080-45c4-8565-08485a5ab41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:58.693919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433d9eb-3080-45c4-8565-08485a5ab41b None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:58.694198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433d9eb-3080-45c4-8565-08485a5ab41b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:58.694473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433d9eb-3080-45c4-8565-08485a5ab41b None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:58.695114 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433d9eb-3080-45c4-8565-08485a5ab41b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:58.696545 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433d9eb-3080-45c4-8565-08485a5ab41b None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:58.701235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c2778-d113-4110-9285-c2aa54c5d03f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:16:58.706633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-001c2778-d113-4110-9285-c2aa54c5d03f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:16:58.706633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:16:58.706633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:16:58.706633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:16:58.706633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:16:58.706633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:16:58.706633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:16:58.706633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:16:58.706633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:16:58.706633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:16:58.706633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:16:58.706633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:16:58.706633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:16:58.706633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:16:58.706633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:16:58.706633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:16:58.707649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:16:58.707649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:16:58.707649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:16:58.707649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:16:58.707649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:16:58.707649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:16:58.707649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:16:58.707649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:16:58.707649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:16:58.707649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:16:58.707649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:16:58.707649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:16:58.707649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:16:58.707649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:16:58.707649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:16:58.707649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:16:58.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:16:58.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:16:58.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:16:58.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:16:58.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:16:58.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:16:58.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:16:58.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:16:58.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:16:58.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:16:58.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:16:58.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:16:58.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 42/84] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:16:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:16:58.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c2778-d113-4110-9285-c2aa54c5d03f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:58.761519 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:58.798340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 43/85] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 19 10:16:58 2022] POST /identity/v3/users => generated 289 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:16:58.799128 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433d9eb-3080-45c4-8565-08485a5ab41b None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:58.824617 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c2778-d113-4110-9285-c2aa54c5d03f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:58.865006 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55ef01-2c2b-4550-873c-62373d0a0651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:58.865796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55ef01-2c2b-4550-873c-62373d0a0651 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:58.866102 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55ef01-2c2b-4550-873c-62373d0a0651 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:58.866413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55ef01-2c2b-4550-873c-62373d0a0651 None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:58.867623 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55ef01-2c2b-4550-873c-62373d0a0651 None admin] RBAC: Authorizing `identity:get_project(project_id=8959cf36bec74b8ca22049270ee47cb2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:58.869130 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55ef01-2c2b-4550-873c-62373d0a0651 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:58.871033 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 43/86] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:16:58 2022] GET /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:58.871663 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55ef01-2c2b-4550-873c-62373d0a0651 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:58.922400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433d9eb-3080-45c4-8565-08485a5ab41b None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:58.951003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:58.960360 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab2cfd-cfb9-4893-9c42-6a578b7708d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:58.961009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab2cfd-cfb9-4893-9c42-6a578b7708d5 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:58.961316 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab2cfd-cfb9-4893-9c42-6a578b7708d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:58.961606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab2cfd-cfb9-4893-9c42-6a578b7708d5 None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:58.962762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab2cfd-cfb9-4893-9c42-6a578b7708d5 None admin] RBAC: Authorizing `identity:get_project(project_id=8959cf36bec74b8ca22049270ee47cb2)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:58.964430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab2cfd-cfb9-4893-9c42-6a578b7708d5 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:58.971636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 44/87] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:16:58 2022] GET /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:58.971636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab2cfd-cfb9-4893-9c42-6a578b7708d5 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:58.994617 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55ef01-2c2b-4550-873c-62373d0a0651 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:59.010410 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:59.033546 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba1e87-d72f-43ec-8e63-520d0f280b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:59.034173 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba1e87-d72f-43ec-8e63-520d0f280b91 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:59.034452 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba1e87-d72f-43ec-8e63-520d0f280b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:59.034726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba1e87-d72f-43ec-8e63-520d0f280b91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:59.097636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab2cfd-cfb9-4893-9c42-6a578b7708d5 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:59.122452 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba1e87-d72f-43ec-8e63-520d0f280b91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:59.123944 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba1e87-d72f-43ec-8e63-520d0f280b91 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:59.143441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156504a-974a-482f-a64d-176898eb4632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:59.144173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156504a-974a-482f-a64d-176898eb4632 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:59.144463 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156504a-974a-482f-a64d-176898eb4632 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:59.144741 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156504a-974a-482f-a64d-176898eb4632 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:59.145441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156504a-974a-482f-a64d-176898eb4632 None admin] RBAC: Adding query filter params (name=member) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:16:59.145752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156504a-974a-482f-a64d-176898eb4632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:59.147149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156504a-974a-482f-a64d-176898eb4632 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:59.155943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 45/88] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:16:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:59.157725 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156504a-974a-482f-a64d-176898eb4632 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:59.230002 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:59.247103 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 44/89] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:16:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:16:59.247968 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba1e87-d72f-43ec-8e63-520d0f280b91 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:59.264241 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:59.285488 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156504a-974a-482f-a64d-176898eb4632 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:59.321280 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd96e2-b6a0-4279-9827-8b5bb61b7597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:59.322242 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd96e2-b6a0-4279-9827-8b5bb61b7597 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:59.322763 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd96e2-b6a0-4279-9827-8b5bb61b7597 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:59.323262 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd96e2-b6a0-4279-9827-8b5bb61b7597 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:59.324183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd96e2-b6a0-4279-9827-8b5bb61b7597 None admin] RBAC: Adding query filter params (role.id=c44d082d88894b888e94797acf7655b7, scope.project.id=8959cf36bec74b8ca22049270ee47cb2, user.id=fda9ca2d3fb54751b2eb0636906d0c8c) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:16:59.324711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd96e2-b6a0-4279-9827-8b5bb61b7597 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:59.326399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd96e2-b6a0-4279-9827-8b5bb61b7597 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:59.333191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 46/90] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:16:59 2022] GET /identity/v3/role_assignments?user.id=fda9ca2d3fb54751b2eb0636906d0c8c&scope.project.id=8959cf36bec74b8ca22049270ee47cb2&role.id=c44d082d88894b888e94797acf7655b7 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:59.341089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd96e2-b6a0-4279-9827-8b5bb61b7597 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:59.357841 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba1e87-d72f-43ec-8e63-520d0f280b91 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:59.395591 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d761d-c537-45bc-a8a9-43024c2a2b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:59.396215 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d761d-c537-45bc-a8a9-43024c2a2b5b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:59.396523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d761d-c537-45bc-a8a9-43024c2a2b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:59.396768 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d761d-c537-45bc-a8a9-43024c2a2b5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:59.402293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64d761d-c537-45bc-a8a9-43024c2a2b5b None admin] RBAC: Adding query filter params (role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.project.id=8959cf36bec74b8ca22049270ee47cb2, user.id=c847b76e4e6649e18dc342c5d998066b) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:16:59.402293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64d761d-c537-45bc-a8a9-43024c2a2b5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:59.402293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64d761d-c537-45bc-a8a9-43024c2a2b5b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:59.407174 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 45/91] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:16:59 2022] GET /identity/v3/role_assignments?user.id=c847b76e4e6649e18dc342c5d998066b&scope.project.id=8959cf36bec74b8ca22049270ee47cb2&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:59.407973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d761d-c537-45bc-a8a9-43024c2a2b5b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:59.584510 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd96e2-b6a0-4279-9827-8b5bb61b7597 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:59.612736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:59.631134 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3d021-fbad-4d5f-b632-30cd3f4d6156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:59.632005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3d021-fbad-4d5f-b632-30cd3f4d6156 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:59.632296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3d021-fbad-4d5f-b632-30cd3f4d6156 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:59.632679 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3d021-fbad-4d5f-b632-30cd3f4d6156 None admin] PATH_INFO: `/v3/users/fda9ca2d3fb54751b2eb0636906d0c8c` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:59.633989 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3d021-fbad-4d5f-b632-30cd3f4d6156 None admin] RBAC: Authorizing `identity:get_user(user_id=fda9ca2d3fb54751b2eb0636906d0c8c)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:59.635625 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3d021-fbad-4d5f-b632-30cd3f4d6156 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:59.637680 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 47/92] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:16:59 2022] GET /identity/v3/users/fda9ca2d3fb54751b2eb0636906d0c8c => generated 275 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:59.644026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3d021-fbad-4d5f-b632-30cd3f4d6156 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:59.653674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:16:59.705761 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d761d-c537-45bc-a8a9-43024c2a2b5b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:59.781135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c650c1-d9ea-4f77-adce-fc9c4f397f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:59.784099 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3d021-fbad-4d5f-b632-30cd3f4d6156 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:16:59.784822 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c650c1-d9ea-4f77-adce-fc9c4f397f93 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:59.785344 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c650c1-d9ea-4f77-adce-fc9c4f397f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:59.785737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c650c1-d9ea-4f77-adce-fc9c4f397f93 None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:16:59.788426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c650c1-d9ea-4f77-adce-fc9c4f397f93 None admin] RBAC: Authorizing `identity:get_project(project_id=8959cf36bec74b8ca22049270ee47cb2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:16:59.790935 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c650c1-d9ea-4f77-adce-fc9c4f397f93 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:16:59.797871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 46/93] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:16:59 2022] GET /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2 => generated 298 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:16:59.799093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c650c1-d9ea-4f77-adce-fc9c4f397f93 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:16:59.882740 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ae55c-66fb-4ae7-bcc1-ead7426f21a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:16:59.883335 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ae55c-66fb-4ae7-bcc1-ead7426f21a0 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:16:59.884663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ae55c-66fb-4ae7-bcc1-ead7426f21a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:16:59.884663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ae55c-66fb-4ae7-bcc1-ead7426f21a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:00.002974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c650c1-d9ea-4f77-adce-fc9c4f397f93 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:00.034755 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:00.041492 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd01000-d727-45d4-8492-c32bd0f09000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:00.042104 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd01000-d727-45d4-8492-c32bd0f09000 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:00.042533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd01000-d727-45d4-8492-c32bd0f09000 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:00.042824 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd01000-d727-45d4-8492-c32bd0f09000 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:00.043557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd01000-d727-45d4-8492-c32bd0f09000 None admin] RBAC: Adding query filter params (role.id=e4309dcb11f74e10ba108b7b00264476, scope.project.id=8959cf36bec74b8ca22049270ee47cb2, user.id=fda9ca2d3fb54751b2eb0636906d0c8c) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:00.043870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd01000-d727-45d4-8492-c32bd0f09000 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:00.045800 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd01000-d727-45d4-8492-c32bd0f09000 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:00.052643 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 47/94] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:16:59 2022] GET /identity/v3/role_assignments?user.id=fda9ca2d3fb54751b2eb0636906d0c8c&scope.project.id=8959cf36bec74b8ca22049270ee47cb2&role.id=e4309dcb11f74e10ba108b7b00264476 => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:00.062341 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ae55c-66fb-4ae7-bcc1-ead7426f21a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:00.063886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ae55c-66fb-4ae7-bcc1-ead7426f21a0 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:00.211443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 48/95] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:16:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:00.217978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd01000-d727-45d4-8492-c32bd0f09000 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:00.239543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:00.342431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd01000-d727-45d4-8492-c32bd0f09000 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:00.377442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f75e6-e8da-4a7b-b776-1b0c62f52df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:00.378305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f75e6-e8da-4a7b-b776-1b0c62f52df2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:00.378731 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f75e6-e8da-4a7b-b776-1b0c62f52df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:00.379184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f75e6-e8da-4a7b-b776-1b0c62f52df2 None admin] PATH_INFO: `/v3/projects/570be56d739e4e869a2dc14432b5ec6c` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:00.380463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f75e6-e8da-4a7b-b776-1b0c62f52df2 None admin] RBAC: Authorizing `identity:get_project(project_id=570be56d739e4e869a2dc14432b5ec6c)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:00.382098 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f75e6-e8da-4a7b-b776-1b0c62f52df2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:00.384276 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 48/96] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:00 2022] GET /identity/v3/projects/570be56d739e4e869a2dc14432b5ec6c => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:00.394177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ae55c-66fb-4ae7-bcc1-ead7426f21a0 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:00.526921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:00.537079 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ae55c-66fb-4ae7-bcc1-ead7426f21a0 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:00.582552 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ca460-9aeb-44b1-9fd1-e5dee066d10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:00.583173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ca460-9aeb-44b1-9fd1-e5dee066d10b None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:00.583459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ca460-9aeb-44b1-9fd1-e5dee066d10b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:00.583736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ca460-9aeb-44b1-9fd1-e5dee066d10b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:00.584481 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ca460-9aeb-44b1-9fd1-e5dee066d10b None admin] RBAC: Adding query filter params (role.id=c44d082d88894b888e94797acf7655b7, scope.project.id=570be56d739e4e869a2dc14432b5ec6c, user.id=fda9ca2d3fb54751b2eb0636906d0c8c) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:00.584793 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ca460-9aeb-44b1-9fd1-e5dee066d10b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:00.586285 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ca460-9aeb-44b1-9fd1-e5dee066d10b None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:00.593757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 49/97] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:17:00 2022] GET /identity/v3/role_assignments?user.id=fda9ca2d3fb54751b2eb0636906d0c8c&scope.project.id=570be56d739e4e869a2dc14432b5ec6c&role.id=c44d082d88894b888e94797acf7655b7 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:01.919178 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9272fb-4166-475c-a570-9583c3fc5340 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:01.919688 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9272fb-4166-475c-a570-9583c3fc5340 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:01.919762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9272fb-4166-475c-a570-9583c3fc5340 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:01.921175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 49/98] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:01.927770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4533d7-70a1-4b32-aee2-f922950f5353 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:01.928296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4533d7-70a1-4b32-aee2-f922950f5353 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:01.928296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4533d7-70a1-4b32-aee2-f922950f5353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:01.957307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-dc4533d7-70a1-4b32-aee2-f922950f5353 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:01.958659 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-dc4533d7-70a1-4b32-aee2-f922950f5353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:02.091493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 50/99] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:02.157263 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab407a54-6169-4bd3-b406-d0a22b286e3b None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:02.157573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab407a54-6169-4bd3-b406-d0a22b286e3b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:02.157827 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab407a54-6169-4bd3-b406-d0a22b286e3b None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:02.159400 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 50/100] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:02.164585 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc8ba2-8190-406c-afe6-791455cb69d5 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:02.165110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc8ba2-8190-406c-afe6-791455cb69d5 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:02.165598 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc8ba2-8190-406c-afe6-791455cb69d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:02.194031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-d4fc8ba2-8190-406c-afe6-791455cb69d5 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:02.195556 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d4fc8ba2-8190-406c-afe6-791455cb69d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:02.284976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 51/101] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:02.518825 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab407a54-6169-4bd3-b406-d0a22b286e3b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:02.603695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:02.627238 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab407a54-6169-4bd3-b406-d0a22b286e3b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:02.662629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503a3f0-6afc-4366-b851-ab1877525f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:02.663256 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503a3f0-6afc-4366-b851-ab1877525f2b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:02.663592 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503a3f0-6afc-4366-b851-ab1877525f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:02.663874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503a3f0-6afc-4366-b851-ab1877525f2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:02.664986 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503a3f0-6afc-4366-b851-ab1877525f2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:02.666454 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503a3f0-6afc-4366-b851-ab1877525f2b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:02.668074 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 51/102] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:17:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:02.674132 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc8ba2-8190-406c-afe6-791455cb69d5 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:02.696484 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:02.723187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c62a83-9860-4cc3-924d-6877ae1b058b None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:02.723490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c62a83-9860-4cc3-924d-6877ae1b058b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:02.723756 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c62a83-9860-4cc3-924d-6877ae1b058b None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:02.725344 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 52/103] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:02.730282 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae84908-416f-4173-9783-fe6136f21eba None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:02.730577 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae84908-416f-4173-9783-fe6136f21eba None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:02.730865 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae84908-416f-4173-9783-fe6136f21eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:02.751414 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-3ae84908-416f-4173-9783-fe6136f21eba None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:02.752618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3ae84908-416f-4173-9783-fe6136f21eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:02.834407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 53/104] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:17:02.835294 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae84908-416f-4173-9783-fe6136f21eba None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:02.866007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc8ba2-8190-406c-afe6-791455cb69d5 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:02.954621 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c694b-3742-4218-bfae-f77e00b844e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:02.955269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c694b-3742-4218-bfae-f77e00b844e8 None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:02.955510 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c694b-3742-4218-bfae-f77e00b844e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:02.955786 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c694b-3742-4218-bfae-f77e00b844e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:02.956449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c694b-3742-4218-bfae-f77e00b844e8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:02.957927 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c694b-3742-4218-bfae-f77e00b844e8 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:03.002104 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:03.033338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae84908-416f-4173-9783-fe6136f21eba None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:03.059767 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 52/105] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 19 10:17:02 2022] POST /identity/v3/projects => generated 302 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:17:03.061050 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c57fe8-ba2b-4659-b2e8-73dbd2914acd None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:03.061387 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c57fe8-ba2b-4659-b2e8-73dbd2914acd None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:03.061667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c57fe8-ba2b-4659-b2e8-73dbd2914acd None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:03.063059 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 53/106] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:03.072860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7b5c8-9573-463c-99e9-de6040e679b3 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:03.073188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7b5c8-9573-463c-99e9-de6040e679b3 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:03.073535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7b5c8-9573-463c-99e9-de6040e679b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:03.083372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0786b-3b1b-4144-92e7-18f08df99111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:03.084035 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0786b-3b1b-4144-92e7-18f08df99111 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:03.084320 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0786b-3b1b-4144-92e7-18f08df99111 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:03.084600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0786b-3b1b-4144-92e7-18f08df99111 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:03.098542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c0786b-3b1b-4144-92e7-18f08df99111 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:03.104636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-d7c0786b-3b1b-4144-92e7-18f08df99111 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 10:17:03.104636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:03.104636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:03.104636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:03.104636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:03.104636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:03.104636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:03.104636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:03.104636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:03.104636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:03.104636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:03.104636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:03.104636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:03.104636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:03.104636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:03.104636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:03.105573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:03.105573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:03.105573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:03.105573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:03.105573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:03.105573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:03.105573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:03.105573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:03.105573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:03.105573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:03.105573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:03.105573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:03.105573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:03.105573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:03.105573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:03.105573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:03.106342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:03.106342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:03.106342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:03.106342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:03.106342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:03.106342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:03.106342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:03.106342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:03.106342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:03.106342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:03.106342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 10:17:03.106342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:03.106342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 54/107] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:03.112617 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0786b-3b1b-4144-92e7-18f08df99111 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:03.117224 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-88a7b5c8-9573-463c-99e9-de6040e679b3 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:03.118485 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-88a7b5c8-9573-463c-99e9-de6040e679b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:03.256347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:03.263088 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0786b-3b1b-4144-92e7-18f08df99111 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:03.267661 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 54/108] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:17:03.268908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7b5c8-9573-463c-99e9-de6040e679b3 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:03.332668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2a0ab-cb59-46d2-a8fa-1e6b0115d1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:03.333235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2a0ab-cb59-46d2-a8fa-1e6b0115d1ad None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:03.333520 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2a0ab-cb59-46d2-a8fa-1e6b0115d1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:03.333796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2a0ab-cb59-46d2-a8fa-1e6b0115d1ad None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:03.334470 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2a0ab-cb59-46d2-a8fa-1e6b0115d1ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:03.334779 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2a0ab-cb59-46d2-a8fa-1e6b0115d1ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:03.336168 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2a0ab-cb59-46d2-a8fa-1e6b0115d1ad None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:03.344379 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:03.363353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 55/109] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:03.374160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2a0ab-cb59-46d2-a8fa-1e6b0115d1ad None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:03.411831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7b5c8-9573-463c-99e9-de6040e679b3 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:03.490667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0a4ef-b79e-4c91-976b-f8d25b1a17c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:03.493238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0a4ef-b79e-4c91-976b-f8d25b1a17c2 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:03.493525 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0a4ef-b79e-4c91-976b-f8d25b1a17c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:03.493796 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0a4ef-b79e-4c91-976b-f8d25b1a17c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:03.506132 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d0a4ef-b79e-4c91-976b-f8d25b1a17c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:03.511736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-a4d0a4ef-b79e-4c91-976b-f8d25b1a17c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:03.511736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:03.511736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:03.511736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:03.511736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:03.511736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:03.511736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:03.511736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:03.511736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:03.511736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:03.511736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:03.511736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:03.511736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:03.511736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:03.511736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:03.511736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:03.513043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:03.513043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:03.513043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:03.513043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:03.513043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:03.513043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:03.513043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:03.513043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:03.513043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:03.513043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:03.513043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:03.513043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:03.513043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:03.513043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:03.513043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:03.513043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:03.513850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:03.513850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:03.513850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:03.513850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:03.513850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:03.513850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:03.513850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:03.513850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:03.513850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:03.513850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:03.513850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:03.513850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:03.513850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 55/110] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:03.514922 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546efc8-247d-4682-9a43-707c36cb415b None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:03.515160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546efc8-247d-4682-9a43-707c36cb415b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:03.515971 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546efc8-247d-4682-9a43-707c36cb415b None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:03.521095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 56/111] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:03.527128 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9329a0-6837-42d5-8fd8-8719f346b198 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:03.527422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9329a0-6837-42d5-8fd8-8719f346b198 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:03.527711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9329a0-6837-42d5-8fd8-8719f346b198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:03.536249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2a0ab-cb59-46d2-a8fa-1e6b0115d1ad None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:03.550797 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-4b9329a0-6837-42d5-8fd8-8719f346b198 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:03.552120 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4b9329a0-6837-42d5-8fd8-8719f346b198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:03.579563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6606f06-c126-4599-86c2-4244c91d186c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:03.580345 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6606f06-c126-4599-86c2-4244c91d186c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:03.580701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6606f06-c126-4599-86c2-4244c91d186c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:03.581051 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6606f06-c126-4599-86c2-4244c91d186c None admin] PATH_INFO: `/v3/users/57908f89930446f486457c86720dd372` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:03.604190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6606f06-c126-4599-86c2-4244c91d186c None admin] RBAC: Authorizing `identity:get_user(user_id=57908f89930446f486457c86720dd372)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:03.604190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6606f06-c126-4599-86c2-4244c91d186c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:03.604190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 56/112] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:03 2022] GET /identity/v3/users/57908f89930446f486457c86720dd372 => generated 289 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:03.604803 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6606f06-c126-4599-86c2-4244c91d186c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:03.631083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:03.649946 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 57/113] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:03.651288 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9329a0-6837-42d5-8fd8-8719f346b198 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:03.675661 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:03.729944 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6606f06-c126-4599-86c2-4244c91d186c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:03.756376 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9329a0-6837-42d5-8fd8-8719f346b198 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:03.769651 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555f3a2-57ac-4c69-8fb3-7daf73858d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:03.770483 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555f3a2-57ac-4c69-8fb3-7daf73858d62 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:03.770902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555f3a2-57ac-4c69-8fb3-7daf73858d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:03.771324 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555f3a2-57ac-4c69-8fb3-7daf73858d62 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:03.772153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555f3a2-57ac-4c69-8fb3-7daf73858d62 None admin] RBAC: Adding query filter params (name=member) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:03.772869 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555f3a2-57ac-4c69-8fb3-7daf73858d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:03.775195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555f3a2-57ac-4c69-8fb3-7daf73858d62 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:03.785936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 57/114] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:03.787069 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555f3a2-57ac-4c69-8fb3-7daf73858d62 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:03.795743 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc190761-72c5-4b43-8903-ba6ac30e06b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:03.796521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc190761-72c5-4b43-8903-ba6ac30e06b9 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:03.797591 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc190761-72c5-4b43-8903-ba6ac30e06b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:03.798310 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc190761-72c5-4b43-8903-ba6ac30e06b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:03.808685 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc190761-72c5-4b43-8903-ba6ac30e06b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:03.816409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-bc190761-72c5-4b43-8903-ba6ac30e06b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:17:03.816409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:03.816409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:03.816409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:03.816409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:03.816409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:03.816409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:03.816409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:03.816409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:03.816409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:03.816409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:03.816409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:03.816409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:03.816409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:03.816409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:03.816409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:03.817499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:03.817499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:03.817499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:03.817499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:03.817499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:03.817499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:03.817499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:03.817499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:03.817499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:03.817499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:03.817499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:03.817499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:03.817499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:03.817499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:03.817499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:03.817499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:03.818300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:03.818300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:03.818300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:03.818300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:03.818300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:03.818300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:03.818300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:03.818300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:03.818300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:03.818300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:03.818300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:17:03.818300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:03.823156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 58/115] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:17:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:03.824484 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046ba9a-3daa-4370-b713-cb76123cb0ff None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:03.824834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046ba9a-3daa-4370-b713-cb76123cb0ff None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:03.825130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046ba9a-3daa-4370-b713-cb76123cb0ff None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:03.826845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 59/116] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:03.827658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046ba9a-3daa-4370-b713-cb76123cb0ff None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:03.879184 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:03.902844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555f3a2-57ac-4c69-8fb3-7daf73858d62 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:03.930447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:03.936123 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046ba9a-3daa-4370-b713-cb76123cb0ff None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:03.942925 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a0973-55a0-4ddf-9fc7-d2724e2bedd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:03.943536 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a0973-55a0-4ddf-9fc7-d2724e2bedd4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:03.943813 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a0973-55a0-4ddf-9fc7-d2724e2bedd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:03.944088 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a0973-55a0-4ddf-9fc7-d2724e2bedd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:03.972790 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72e89a-393d-43b4-a7f3-9fcc0f4368b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:03.973497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72e89a-393d-43b4-a7f3-9fcc0f4368b6 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:03.973736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72e89a-393d-43b4-a7f3-9fcc0f4368b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:03.974013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72e89a-393d-43b4-a7f3-9fcc0f4368b6 None admin] PATH_INFO: `/v3/users/fda9ca2d3fb54751b2eb0636906d0c8c` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:03.975146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72e89a-393d-43b4-a7f3-9fcc0f4368b6 None admin] RBAC: Authorizing `identity:get_user(user_id=fda9ca2d3fb54751b2eb0636906d0c8c)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:03.976664 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72e89a-393d-43b4-a7f3-9fcc0f4368b6 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:03.981178 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 60/117] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:03 2022] GET /identity/v3/users/fda9ca2d3fb54751b2eb0636906d0c8c => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:03.981178 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72e89a-393d-43b4-a7f3-9fcc0f4368b6 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:04.018254 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a0973-55a0-4ddf-9fc7-d2724e2bedd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:04.019861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a0973-55a0-4ddf-9fc7-d2724e2bedd4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:04.088011 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72e89a-393d-43b4-a7f3-9fcc0f4368b6 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:04.104711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:04.126885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e013e-3a0f-4f7c-ac19-3a0e9e24a2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:04.127714 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e013e-3a0f-4f7c-ac19-3a0e9e24a2af None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:04.128008 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e013e-3a0f-4f7c-ac19-3a0e9e24a2af None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:04.128532 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e013e-3a0f-4f7c-ac19-3a0e9e24a2af None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:04.129650 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e013e-3a0f-4f7c-ac19-3a0e9e24a2af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:04.129971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 58/118] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:04.130783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e013e-3a0f-4f7c-ac19-3a0e9e24a2af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:04.132185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e013e-3a0f-4f7c-ac19-3a0e9e24a2af None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:04.135784 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b408db-c11f-4c20-9372-581e51348502 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:04.136078 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b408db-c11f-4c20-9372-581e51348502 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:04.136356 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b408db-c11f-4c20-9372-581e51348502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:04.141755 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 61/119] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:04.141755 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e013e-3a0f-4f7c-ac19-3a0e9e24a2af None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:04.160222 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c4b408db-c11f-4c20-9372-581e51348502 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:04.161455 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c4b408db-c11f-4c20-9372-581e51348502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:04.172918 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:04.233640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 59/120] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:04.234528 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b408db-c11f-4c20-9372-581e51348502 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:04.247357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e013e-3a0f-4f7c-ac19-3a0e9e24a2af None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:04.282768 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334dbc7-d0fd-4c15-a13e-ce0ef563a422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:04.283400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334dbc7-d0fd-4c15-a13e-ce0ef563a422 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:04.283679 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334dbc7-d0fd-4c15-a13e-ce0ef563a422 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:04.283953 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334dbc7-d0fd-4c15-a13e-ce0ef563a422 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:04.294936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334dbc7-d0fd-4c15-a13e-ce0ef563a422 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:04.300899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-a334dbc7-d0fd-4c15-a13e-ce0ef563a422 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:04.300899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:04.300899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:04.300899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:04.300899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:04.300899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:04.300899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:04.300899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:04.300899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:04.300899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:04.300899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:04.300899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:04.300899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:04.300899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:04.300899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:04.300899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:04.302026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:04.302026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:04.302026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:04.302026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:04.302026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:04.302026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:04.302026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:04.302026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:04.302026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:04.302026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:04.302026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:04.302026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:04.302026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:04.302026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:04.302026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:04.302026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:04.303185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:04.303185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:04.303185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:04.303185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:04.303185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:04.303185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:04.303185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:04.303185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:04.303185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:04.303185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:04.303185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:04.303185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:04.303185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 62/121] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:17:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:04.303185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334dbc7-d0fd-4c15-a13e-ce0ef563a422 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:04.344083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:04.367845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b408db-c11f-4c20-9372-581e51348502 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:04.406630 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2efb6d-858a-404e-80ea-55ac1820fd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:04.407241 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2efb6d-858a-404e-80ea-55ac1820fd06 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:04.407525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2efb6d-858a-404e-80ea-55ac1820fd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:04.407802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2efb6d-858a-404e-80ea-55ac1820fd06 None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:04.408938 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2efb6d-858a-404e-80ea-55ac1820fd06 None admin] RBAC: Authorizing `identity:get_project(project_id=8959cf36bec74b8ca22049270ee47cb2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:04.410413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2efb6d-858a-404e-80ea-55ac1820fd06 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:04.412105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 60/122] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:04 2022] GET /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:04.412881 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2efb6d-858a-404e-80ea-55ac1820fd06 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:04.433251 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334dbc7-d0fd-4c15-a13e-ce0ef563a422 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:04.434473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:04.480722 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d18dd6-fe49-4786-9d66-1eaad81bf05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:04.481747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d18dd6-fe49-4786-9d66-1eaad81bf05e None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:04.482449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d18dd6-fe49-4786-9d66-1eaad81bf05e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:04.482896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d18dd6-fe49-4786-9d66-1eaad81bf05e None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:04.485623 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d18dd6-fe49-4786-9d66-1eaad81bf05e None admin] RBAC: Authorizing `identity:get_project(project_id=8959cf36bec74b8ca22049270ee47cb2)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:04.488159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d18dd6-fe49-4786-9d66-1eaad81bf05e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:04.490967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 63/123] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:04 2022] GET /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:04.492005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d18dd6-fe49-4786-9d66-1eaad81bf05e None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:04.541980 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2efb6d-858a-404e-80ea-55ac1820fd06 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:04.581522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3518e-8141-4cbd-a701-3dc6f23e81fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:04.582260 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3518e-8141-4cbd-a701-3dc6f23e81fe None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:04.582542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3518e-8141-4cbd-a701-3dc6f23e81fe None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:04.582819 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3518e-8141-4cbd-a701-3dc6f23e81fe None admin] PATH_INFO: `/v3/users/c847b76e4e6649e18dc342c5d998066b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:04.583935 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d3518e-8141-4cbd-a701-3dc6f23e81fe None admin] RBAC: Authorizing `identity:get_user(user_id=c847b76e4e6649e18dc342c5d998066b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:04.585486 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d3518e-8141-4cbd-a701-3dc6f23e81fe None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:04.587239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 61/124] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:04 2022] GET /identity/v3/users/c847b76e4e6649e18dc342c5d998066b => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:04.587799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3518e-8141-4cbd-a701-3dc6f23e81fe None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:04.610976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d18dd6-fe49-4786-9d66-1eaad81bf05e None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:04.614265 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:04.649617 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac702333-652f-46bd-ba17-67a2710aaa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:04.650651 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac702333-652f-46bd-ba17-67a2710aaa12 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:04.651211 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac702333-652f-46bd-ba17-67a2710aaa12 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:04.651511 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac702333-652f-46bd-ba17-67a2710aaa12 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:04.652329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac702333-652f-46bd-ba17-67a2710aaa12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:04.652635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac702333-652f-46bd-ba17-67a2710aaa12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:04.654038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac702333-652f-46bd-ba17-67a2710aaa12 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:04.662762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 64/125] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:17:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:04.666024 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac702333-652f-46bd-ba17-67a2710aaa12 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:04.671655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:04.690750 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3518e-8141-4cbd-a701-3dc6f23e81fe None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:04.727413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155f426-3a6c-4ac5-8355-f5879cf81ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:04.728068 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155f426-3a6c-4ac5-8355-f5879cf81ccf None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:04.728764 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155f426-3a6c-4ac5-8355-f5879cf81ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:04.729042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155f426-3a6c-4ac5-8355-f5879cf81ccf None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2/users/fda9ca2d3fb54751b2eb0636906d0c8c/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:04.731131 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155f426-3a6c-4ac5-8355-f5879cf81ccf None admin] RBAC: Authorizing `identity:create_grant(project_id=8959cf36bec74b8ca22049270ee47cb2, user_id=fda9ca2d3fb54751b2eb0636906d0c8c, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:04.732908 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155f426-3a6c-4ac5-8355-f5879cf81ccf None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:04.774092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac702333-652f-46bd-ba17-67a2710aaa12 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:04.806615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5a6c5-bc76-4315-aa2a-72969cbf23e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:04.807218 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5a6c5-bc76-4315-aa2a-72969cbf23e6 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:04.807504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5a6c5-bc76-4315-aa2a-72969cbf23e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:04.807780 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5a6c5-bc76-4315-aa2a-72969cbf23e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:04.808505 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5a6c5-bc76-4315-aa2a-72969cbf23e6 None admin] RBAC: Adding query filter params (role.id=c1c0bce9f7a34def89550deb2e26113a, scope.project.id=8959cf36bec74b8ca22049270ee47cb2, user.id=57908f89930446f486457c86720dd372) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:04.808818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5a6c5-bc76-4315-aa2a-72969cbf23e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:04.810298 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5a6c5-bc76-4315-aa2a-72969cbf23e6 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:04.821495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 65/126] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:17:04 2022] GET /identity/v3/role_assignments?user.id=57908f89930446f486457c86720dd372&scope.project.id=8959cf36bec74b8ca22049270ee47cb2&role.id=c1c0bce9f7a34def89550deb2e26113a => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:04.821495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5a6c5-bc76-4315-aa2a-72969cbf23e6 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:04.824016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 62/127] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:17:04 2022] PUT /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2/users/fda9ca2d3fb54751b2eb0636906d0c8c/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:17:04.825270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155f426-3a6c-4ac5-8355-f5879cf81ccf None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:04.900232 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:05.050348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5a6c5-bc76-4315-aa2a-72969cbf23e6 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:05.056975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155f426-3a6c-4ac5-8355-f5879cf81ccf None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:05.097026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f857d-66c1-4caf-be7c-93016822983b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:05.097701 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f857d-66c1-4caf-be7c-93016822983b None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:05.097984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f857d-66c1-4caf-be7c-93016822983b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:05.098259 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f857d-66c1-4caf-be7c-93016822983b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:05.102768 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab121a-a1fb-4630-8c03-a4f25809fd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:05.103630 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab121a-a1fb-4630-8c03-a4f25809fd69 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:05.104103 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab121a-a1fb-4630-8c03-a4f25809fd69 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:05.104781 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab121a-a1fb-4630-8c03-a4f25809fd69 None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:05.106717 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab121a-a1fb-4630-8c03-a4f25809fd69 None admin] RBAC: Authorizing `identity:get_project(project_id=8959cf36bec74b8ca22049270ee47cb2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:05.108367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab121a-a1fb-4630-8c03-a4f25809fd69 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:05.109953 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f857d-66c1-4caf-be7c-93016822983b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:05.115267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-684f857d-66c1-4caf-be7c-93016822983b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:05.115267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:05.115267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:05.115267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:05.115267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:05.115267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:05.115267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:05.115267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:05.115267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:05.115267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:05.115267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:05.115267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:05.115267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:05.115267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:05.115267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:05.115267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:05.116166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:05.116166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:05.116166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:05.116166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:05.116166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:05.116166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:05.116166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:05.116166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:05.116166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:05.116166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:05.116166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:05.116166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:05.116166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:05.116166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:05.116166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:05.116166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:05.117587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:05.117587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:05.117587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:05.117587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:05.117587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:05.117587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:05.117587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:05.117587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:05.117587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:05.117587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:05.117587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:05.117587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:05.117587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 66/129] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:05.117587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f857d-66c1-4caf-be7c-93016822983b None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:05.118626 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 63/129] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:04 2022] GET /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2 => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:05.121302 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab121a-a1fb-4630-8c03-a4f25809fd69 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:05.134518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:05.260648 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:05.281582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f857d-66c1-4caf-be7c-93016822983b None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:05.300455 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab121a-a1fb-4630-8c03-a4f25809fd69 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:05.326573 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c8729-0e31-4cd9-8d3d-33fcb23fc038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:05.327319 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c8729-0e31-4cd9-8d3d-33fcb23fc038 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:05.327692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c8729-0e31-4cd9-8d3d-33fcb23fc038 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:05.328598 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c8729-0e31-4cd9-8d3d-33fcb23fc038 None admin] PATH_INFO: `/v3/users/fda9ca2d3fb54751b2eb0636906d0c8c` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:05.333025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c8729-0e31-4cd9-8d3d-33fcb23fc038 None admin] RBAC: Authorizing `identity:get_user(user_id=fda9ca2d3fb54751b2eb0636906d0c8c)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:05.334702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c8729-0e31-4cd9-8d3d-33fcb23fc038 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:05.336292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 67/130] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:05 2022] GET /identity/v3/users/fda9ca2d3fb54751b2eb0636906d0c8c => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:05.337187 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c8729-0e31-4cd9-8d3d-33fcb23fc038 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:05.353818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d43cc-82f6-454c-8bc4-2b1325227d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:05.354495 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d43cc-82f6-454c-8bc4-2b1325227d47 None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:05.354774 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d43cc-82f6-454c-8bc4-2b1325227d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:05.355050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d43cc-82f6-454c-8bc4-2b1325227d47 None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2/users/c847b76e4e6649e18dc342c5d998066b/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:05.361821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d43cc-82f6-454c-8bc4-2b1325227d47 None admin] RBAC: Authorizing `identity:create_grant(project_id=8959cf36bec74b8ca22049270ee47cb2, user_id=c847b76e4e6649e18dc342c5d998066b, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:05.364721 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d43cc-82f6-454c-8bc4-2b1325227d47 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:05.462000 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c8729-0e31-4cd9-8d3d-33fcb23fc038 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:05.483755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 64/131] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:17:05 2022] PUT /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2/users/c847b76e4e6649e18dc342c5d998066b/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:17:05.484742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d43cc-82f6-454c-8bc4-2b1325227d47 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:05.529130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f895f1-8613-45ae-a87a-eb9155be5e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:05.529762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f895f1-8613-45ae-a87a-eb9155be5e01 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:05.530042 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f895f1-8613-45ae-a87a-eb9155be5e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:05.532202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f895f1-8613-45ae-a87a-eb9155be5e01 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:05.533151 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f895f1-8613-45ae-a87a-eb9155be5e01 None admin] RBAC: Adding query filter params (name=member) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:05.534410 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f895f1-8613-45ae-a87a-eb9155be5e01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:05.535860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f895f1-8613-45ae-a87a-eb9155be5e01 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:05.552997 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 68/132] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:05.559672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f895f1-8613-45ae-a87a-eb9155be5e01 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:05.594514 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:05.634719 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:05.705096 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d43cc-82f6-454c-8bc4-2b1325227d47 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:05.759876 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f895f1-8613-45ae-a87a-eb9155be5e01 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:05.777788 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8135df7-97be-4648-8fb0-1a326e486ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:05.778547 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8135df7-97be-4648-8fb0-1a326e486ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:05.778863 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8135df7-97be-4648-8fb0-1a326e486ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:05.779108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8135df7-97be-4648-8fb0-1a326e486ca0 None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:05.782287 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8135df7-97be-4648-8fb0-1a326e486ca0 None admin] RBAC: Authorizing `identity:get_project(project_id=8959cf36bec74b8ca22049270ee47cb2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:05.783848 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8135df7-97be-4648-8fb0-1a326e486ca0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:05.785671 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 65/133] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:05 2022] GET /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2 => generated 298 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:05.792758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8135df7-97be-4648-8fb0-1a326e486ca0 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:05.813552 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aae189-b1a0-4657-89c1-f640dbd7df06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:05.814185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aae189-b1a0-4657-89c1-f640dbd7df06 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:05.814522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aae189-b1a0-4657-89c1-f640dbd7df06 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:05.814807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aae189-b1a0-4657-89c1-f640dbd7df06 None admin] PATH_INFO: `/v3/users/fda9ca2d3fb54751b2eb0636906d0c8c` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:05.819641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aae189-b1a0-4657-89c1-f640dbd7df06 None admin] RBAC: Authorizing `identity:get_user(user_id=fda9ca2d3fb54751b2eb0636906d0c8c)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:05.821156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aae189-b1a0-4657-89c1-f640dbd7df06 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:05.823107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 69/134] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:05 2022] GET /identity/v3/users/fda9ca2d3fb54751b2eb0636906d0c8c => generated 275 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:05.829009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aae189-b1a0-4657-89c1-f640dbd7df06 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:05.955465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8135df7-97be-4648-8fb0-1a326e486ca0 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:06.002643 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159267e6-3134-4b3e-9a76-e06252299959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:06.003314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159267e6-3134-4b3e-9a76-e06252299959 None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:06.003600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159267e6-3134-4b3e-9a76-e06252299959 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:06.003880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159267e6-3134-4b3e-9a76-e06252299959 None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2/users/fda9ca2d3fb54751b2eb0636906d0c8c/roles/e4309dcb11f74e10ba108b7b00264476` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:06.006133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159267e6-3134-4b3e-9a76-e06252299959 None admin] RBAC: Authorizing `identity:create_grant(project_id=8959cf36bec74b8ca22049270ee47cb2, user_id=fda9ca2d3fb54751b2eb0636906d0c8c, role_id=e4309dcb11f74e10ba108b7b00264476)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:06.007625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159267e6-3134-4b3e-9a76-e06252299959 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:06.017440 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:06.030637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aae189-b1a0-4657-89c1-f640dbd7df06 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:06.066120 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42c71c-0d7f-427e-92a6-0f1ba7357cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:06.066701 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42c71c-0d7f-427e-92a6-0f1ba7357cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:06.066947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42c71c-0d7f-427e-92a6-0f1ba7357cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:06.067221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42c71c-0d7f-427e-92a6-0f1ba7357cc8 None admin] PATH_INFO: `/v3/projects/570be56d739e4e869a2dc14432b5ec6c` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:06.068348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42c71c-0d7f-427e-92a6-0f1ba7357cc8 None admin] RBAC: Authorizing `identity:get_project(project_id=570be56d739e4e869a2dc14432b5ec6c)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:06.069833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42c71c-0d7f-427e-92a6-0f1ba7357cc8 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:06.073018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:06.078365 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 66/135] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:17:05 2022] PUT /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2/users/fda9ca2d3fb54751b2eb0636906d0c8c/roles/e4309dcb11f74e10ba108b7b00264476 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:17:06.079470 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728cd3a-c1ae-442e-b17d-dd0d18ecb570 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:06.081439 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728cd3a-c1ae-442e-b17d-dd0d18ecb570 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:06.081439 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728cd3a-c1ae-442e-b17d-dd0d18ecb570 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:06.081439 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 67/136] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:06.081841 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 70/137] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:05 2022] GET /identity/v3/projects/570be56d739e4e869a2dc14432b5ec6c => generated 312 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:06.087893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42c71c-0d7f-427e-92a6-0f1ba7357cc8 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:06.088588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfde80-09f7-4c6d-ab78-83af1a6451a3 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:06.088936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfde80-09f7-4c6d-ab78-83af1a6451a3 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:06.089313 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfde80-09f7-4c6d-ab78-83af1a6451a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:06.126141 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-72dfde80-09f7-4c6d-ab78-83af1a6451a3 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:06.127367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-72dfde80-09f7-4c6d-ab78-83af1a6451a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:06.266103 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 68/138] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:06.305344 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42c71c-0d7f-427e-92a6-0f1ba7357cc8 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:06.360545 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c19e8-60d9-4453-b4bc-5bac74ca9fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:06.361174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c19e8-60d9-4453-b4bc-5bac74ca9fd2 None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:06.361459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c19e8-60d9-4453-b4bc-5bac74ca9fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:06.361794 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c19e8-60d9-4453-b4bc-5bac74ca9fd2 None admin] PATH_INFO: `/v3/projects/570be56d739e4e869a2dc14432b5ec6c/users/fda9ca2d3fb54751b2eb0636906d0c8c/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:06.365833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c19e8-60d9-4453-b4bc-5bac74ca9fd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=570be56d739e4e869a2dc14432b5ec6c, user_id=fda9ca2d3fb54751b2eb0636906d0c8c, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:06.367287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c19e8-60d9-4453-b4bc-5bac74ca9fd2 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:06.433168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 71/139] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:17:06 2022] PUT /identity/v3/projects/570be56d739e4e869a2dc14432b5ec6c/users/fda9ca2d3fb54751b2eb0636906d0c8c/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:17:06.660144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dfde80-09f7-4c6d-ab78-83af1a6451a3 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:06.772662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dfde80-09f7-4c6d-ab78-83af1a6451a3 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:06.802915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:06.815959 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a5a18-b1bb-4a29-bccf-676b5635ca43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:06.816624 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a5a18-b1bb-4a29-bccf-676b5635ca43 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:06.820580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a5a18-b1bb-4a29-bccf-676b5635ca43 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:06.820580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a5a18-b1bb-4a29-bccf-676b5635ca43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:06.820580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a5a18-b1bb-4a29-bccf-676b5635ca43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:06.820580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a5a18-b1bb-4a29-bccf-676b5635ca43 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:06.821836 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 69/140] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:17:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:06.827399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c19e8-60d9-4453-b4bc-5bac74ca9fd2 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:06.847248 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:06.938046 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c19e8-60d9-4453-b4bc-5bac74ca9fd2 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:06.974025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891e762-a36c-4373-b8be-995901d1d3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:06.974640 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891e762-a36c-4373-b8be-995901d1d3ab None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:06.974925 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891e762-a36c-4373-b8be-995901d1d3ab None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:06.975197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891e762-a36c-4373-b8be-995901d1d3ab None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:06.975874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8891e762-a36c-4373-b8be-995901d1d3ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:06.977366 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8891e762-a36c-4373-b8be-995901d1d3ab None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:07.107987 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 72/141] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 19 10:17:06 2022] POST /identity/v3/users => generated 283 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:17:07.713275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927c0b2-ae09-4373-b3f7-fef95769acc5 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:07.713810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927c0b2-ae09-4373-b3f7-fef95769acc5 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:07.713810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927c0b2-ae09-4373-b3f7-fef95769acc5 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:07.715314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 70/142] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:07.724222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426dfac-2a22-4c56-ba69-4a87e49185e9 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:07.724515 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426dfac-2a22-4c56-ba69-4a87e49185e9 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:07.724789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426dfac-2a22-4c56-ba69-4a87e49185e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:07.746201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-6426dfac-2a22-4c56-ba69-4a87e49185e9 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:07.747468 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-6426dfac-2a22-4c56-ba69-4a87e49185e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:07.823450 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 73/143] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:07.973291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f533d58-0197-4d6c-b484-4702329da3b4 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:07.973590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f533d58-0197-4d6c-b484-4702329da3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:07.973795 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f533d58-0197-4d6c-b484-4702329da3b4 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:07.975266 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 71/144] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:07.990447 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb83483-c792-4795-b584-51ecd3902393 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:07.990735 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb83483-c792-4795-b584-51ecd3902393 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:07.991006 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb83483-c792-4795-b584-51ecd3902393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:08.002942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:08.019248 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-3bb83483-c792-4795-b584-51ecd3902393 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:08.020454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-3bb83483-c792-4795-b584-51ecd3902393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:08.095878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 74/145] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:08.205868 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f533d58-0197-4d6c-b484-4702329da3b4 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:08.326120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f533d58-0197-4d6c-b484-4702329da3b4 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:08.369027 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e953d-bd85-48d8-ac42-db2a0571b351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:08.369718 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e953d-bd85-48d8-ac42-db2a0571b351 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:08.370012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e953d-bd85-48d8-ac42-db2a0571b351 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:08.370297 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e953d-bd85-48d8-ac42-db2a0571b351 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:08.383880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e953d-bd85-48d8-ac42-db2a0571b351 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:08.391107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-137e953d-bd85-48d8-ac42-db2a0571b351 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:08.391107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:08.391107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:08.391107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:08.391107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:08.391107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:08.391107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:08.391107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:08.391107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:08.391107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:08.391107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:08.391107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:08.391107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:08.391107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:08.391107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:08.391107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:08.392063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:08.392063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:08.392063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:08.392063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:08.392063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:08.392063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:08.392063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:08.392063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:08.392063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:08.392063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:08.392063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:08.392063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:08.392063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:08.392063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:08.392063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:08.392063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:08.392831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:08.392831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:08.392831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:08.392831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:08.392831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:08.392831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:08.392831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:08.392831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:08.392831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:08.392831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:08.392831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:08.392831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:08.392831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 72/146] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:08.401301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb83483-c792-4795-b584-51ecd3902393 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:08.517357 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e953d-bd85-48d8-ac42-db2a0571b351 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:08.595856 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb83483-c792-4795-b584-51ecd3902393 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:08.625455 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:08.657128 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:08.667125 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae243fb-bd33-47b1-95f5-5f17e2480cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:08.668953 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae243fb-bd33-47b1-95f5-5f17e2480cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:08.669582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae243fb-bd33-47b1-95f5-5f17e2480cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:08.671044 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae243fb-bd33-47b1-95f5-5f17e2480cd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:08.672424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae243fb-bd33-47b1-95f5-5f17e2480cd2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:08.673091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae243fb-bd33-47b1-95f5-5f17e2480cd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:08.675515 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae243fb-bd33-47b1-95f5-5f17e2480cd2 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:08.690272 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 75/147] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:08.692453 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab58daf-d7d8-4cca-918f-7e6833916310 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:08.694297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab58daf-d7d8-4cca-918f-7e6833916310 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:08.695499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab58daf-d7d8-4cca-918f-7e6833916310 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:08.696983 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 76/148] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:08.699896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab58daf-d7d8-4cca-918f-7e6833916310 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:08.706537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e953d-bd85-48d8-ac42-db2a0571b351 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:08.771876 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3324b-7fb5-40e7-84d0-f329e0529feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:08.773262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3324b-7fb5-40e7-84d0-f329e0529feb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:08.773742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3324b-7fb5-40e7-84d0-f329e0529feb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:08.775997 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3324b-7fb5-40e7-84d0-f329e0529feb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:08.800063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f3324b-7fb5-40e7-84d0-f329e0529feb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:08.805738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-18f3324b-7fb5-40e7-84d0-f329e0529feb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 10:17:08.805738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:08.805738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:08.805738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:08.805738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:08.805738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:08.805738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:08.805738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:08.805738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:08.805738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:08.805738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:08.805738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:08.805738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:08.805738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:08.805738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:08.805738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:08.806595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:08.806595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:08.806595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:08.806595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:08.806595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:08.806595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:08.806595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:08.806595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:08.806595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:08.806595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:08.806595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:08.806595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:08.806595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:08.806595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:08.806595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:08.806595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:08.807370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:08.807370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:08.807370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:08.807370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:08.807370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:08.807370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:08.807370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:08.807370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:08.807370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:08.807370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:08.807370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 10:17:08.807370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:08.815921 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 73/149] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:17:08.818512 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b003e-8201-47de-a08f-c8590de6f7c7 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:08.820211 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b003e-8201-47de-a08f-c8590de6f7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:08.820634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b003e-8201-47de-a08f-c8590de6f7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:08.857419 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f87b003e-8201-47de-a08f-c8590de6f7c7 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:08.858537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f87b003e-8201-47de-a08f-c8590de6f7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:08.912322 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab58daf-d7d8-4cca-918f-7e6833916310 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:08.949053 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 74/150] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:08.950048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b003e-8201-47de-a08f-c8590de6f7c7 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:08.968296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b024fc-54b5-4535-8f01-021a24633eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:08.968946 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b024fc-54b5-4535-8f01-021a24633eba None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:08.969265 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b024fc-54b5-4535-8f01-021a24633eba None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:08.969542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b024fc-54b5-4535-8f01-021a24633eba None admin] PATH_INFO: `/v3/users/fda9ca2d3fb54751b2eb0636906d0c8c` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:08.973779 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b024fc-54b5-4535-8f01-021a24633eba None admin] RBAC: Authorizing `identity:get_user(user_id=fda9ca2d3fb54751b2eb0636906d0c8c)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:08.975333 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b024fc-54b5-4535-8f01-021a24633eba None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:08.977002 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 77/151] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:08 2022] GET /identity/v3/users/fda9ca2d3fb54751b2eb0636906d0c8c => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:08.984402 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b024fc-54b5-4535-8f01-021a24633eba None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:09.049459 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:09.068357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:09.102989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b003e-8201-47de-a08f-c8590de6f7c7 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:09.126115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b024fc-54b5-4535-8f01-021a24633eba None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:09.173031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bda33-ae62-4747-9614-5297d255d756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:09.173669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bda33-ae62-4747-9614-5297d255d756 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:09.173948 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bda33-ae62-4747-9614-5297d255d756 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:09.174217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bda33-ae62-4747-9614-5297d255d756 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:09.174891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bda33-ae62-4747-9614-5297d255d756 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:09.175200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bda33-ae62-4747-9614-5297d255d756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:09.176580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bda33-ae62-4747-9614-5297d255d756 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:09.192251 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 75/152] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:09.195274 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb02ca4-d901-45be-8636-7ac96b3d2b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:09.195897 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb02ca4-d901-45be-8636-7ac96b3d2b09 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:09.196174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb02ca4-d901-45be-8636-7ac96b3d2b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:09.196449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb02ca4-d901-45be-8636-7ac96b3d2b09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:09.198312 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bda33-ae62-4747-9614-5297d255d756 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:09.292577 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb02ca4-d901-45be-8636-7ac96b3d2b09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:09.294091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb02ca4-d901-45be-8636-7ac96b3d2b09 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:09.364859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bda33-ae62-4747-9614-5297d255d756 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:09.371025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:09.380091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:09.422186 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86688f45-a4ed-4f07-a37a-1852f8e97bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:09.423109 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86688f45-a4ed-4f07-a37a-1852f8e97bab None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:09.423624 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86688f45-a4ed-4f07-a37a-1852f8e97bab None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:09.424102 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86688f45-a4ed-4f07-a37a-1852f8e97bab None admin] PATH_INFO: `/v3/users/57908f89930446f486457c86720dd372` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:09.429605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86688f45-a4ed-4f07-a37a-1852f8e97bab None admin] RBAC: Authorizing `identity:get_user(user_id=57908f89930446f486457c86720dd372)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:09.431550 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86688f45-a4ed-4f07-a37a-1852f8e97bab None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:09.433576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 76/153] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:09 2022] GET /identity/v3/users/57908f89930446f486457c86720dd372 => generated 289 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:09.434527 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86688f45-a4ed-4f07-a37a-1852f8e97bab None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:09.451936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 78/154] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:09.454018 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb02ca4-d901-45be-8636-7ac96b3d2b09 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:09.599022 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86688f45-a4ed-4f07-a37a-1852f8e97bab None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:09.651244 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07010be-4d3e-4672-8af3-ea6cc55d4b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:09.651811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07010be-4d3e-4672-8af3-ea6cc55d4b9e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:09.652094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07010be-4d3e-4672-8af3-ea6cc55d4b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:09.652390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07010be-4d3e-4672-8af3-ea6cc55d4b9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:09.663348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb02ca4-d901-45be-8636-7ac96b3d2b09 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:09.670778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07010be-4d3e-4672-8af3-ea6cc55d4b9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:09.676820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-a07010be-4d3e-4672-8af3-ea6cc55d4b9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:17:09.676820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:09.676820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:09.676820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:09.676820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:09.676820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:09.676820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:09.676820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:09.676820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:09.676820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:09.676820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:09.676820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:09.676820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:09.676820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:09.676820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:09.676820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:09.682105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:09.682105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:09.682105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:09.682105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:09.682105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:09.682105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:09.682105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:09.682105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:09.682105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:09.682105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:09.682105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:09.682105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:09.682105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:09.682105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:09.682105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:09.682105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:09.683093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:09.683093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:09.683093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:09.683093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:09.683093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:09.683093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:09.683093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:09.683093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:09.683093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:09.683093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:09.683093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:17:09.683093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:09.685170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 77/155] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:17:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:09.686167 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07010be-4d3e-4672-8af3-ea6cc55d4b9e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:09.712274 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ae255-3933-48e2-aa91-fcff23d9ef7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:09.712929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ae255-3933-48e2-aa91-fcff23d9ef7e None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:09.713244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ae255-3933-48e2-aa91-fcff23d9ef7e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:09.713512 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ae255-3933-48e2-aa91-fcff23d9ef7e None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:09.715105 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ae255-3933-48e2-aa91-fcff23d9ef7e None admin] RBAC: Authorizing `identity:get_project(project_id=8959cf36bec74b8ca22049270ee47cb2)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:09.716720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ae255-3933-48e2-aa91-fcff23d9ef7e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:09.718926 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 79/156] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:09 2022] GET /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:09.723773 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e3c8f-4afb-4677-be9d-755fdcfd19f1 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:09.724050 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e3c8f-4afb-4677-be9d-755fdcfd19f1 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:09.724327 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e3c8f-4afb-4677-be9d-755fdcfd19f1 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:09.725742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 80/157] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:09.728203 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e3c8f-4afb-4677-be9d-755fdcfd19f1 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:09.756702 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:09.787539 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:09.873920 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07010be-4d3e-4672-8af3-ea6cc55d4b9e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:09.890812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e3c8f-4afb-4677-be9d-755fdcfd19f1 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:09.945178 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af315b-0d6b-49c8-b52a-abddb1516fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:09.945814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af315b-0d6b-49c8-b52a-abddb1516fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:09.946096 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af315b-0d6b-49c8-b52a-abddb1516fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:09.946371 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af315b-0d6b-49c8-b52a-abddb1516fd7 None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:09.949617 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af315b-0d6b-49c8-b52a-abddb1516fd7 None admin] RBAC: Authorizing `identity:get_project(project_id=8959cf36bec74b8ca22049270ee47cb2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:09.951076 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af315b-0d6b-49c8-b52a-abddb1516fd7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:09.953271 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 78/158] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:09 2022] GET /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:09.954440 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7b0b0-4ce1-4c38-bda8-3a7104c1ac61 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:09.954676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7b0b0-4ce1-4c38-bda8-3a7104c1ac61 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:09.954945 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7b0b0-4ce1-4c38-bda8-3a7104c1ac61 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:09.958936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 79/159] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:09.960081 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5d05e-88f2-48e0-8c38-fd2c33f7c7e6 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:09.960365 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5d05e-88f2-48e0-8c38-fd2c33f7c7e6 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:09.960641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5d05e-88f2-48e0-8c38-fd2c33f7c7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:09.978939 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac909d5b-1881-4a4a-a77f-6e484d1ff8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:09.979570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac909d5b-1881-4a4a-a77f-6e484d1ff8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:09.979831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac909d5b-1881-4a4a-a77f-6e484d1ff8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:09.980101 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac909d5b-1881-4a4a-a77f-6e484d1ff8d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:09.980765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac909d5b-1881-4a4a-a77f-6e484d1ff8d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:09.981089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac909d5b-1881-4a4a-a77f-6e484d1ff8d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:09.982477 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac909d5b-1881-4a4a-a77f-6e484d1ff8d3 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:09.992935 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-67d5d05e-88f2-48e0-8c38-fd2c33f7c7e6 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:09.994166 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-67d5d05e-88f2-48e0-8c38-fd2c33f7c7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:09.999134 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 81/160] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:10.000549 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac909d5b-1881-4a4a-a77f-6e484d1ff8d3 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:10.110899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 80/161] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:10.111764 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5d05e-88f2-48e0-8c38-fd2c33f7c7e6 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:10.178682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:10.190231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:10.206010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac909d5b-1881-4a4a-a77f-6e484d1ff8d3 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:10.277735 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495a063-44e3-4e40-98e5-573950667c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:10.278407 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495a063-44e3-4e40-98e5-573950667c9e None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:10.278687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495a063-44e3-4e40-98e5-573950667c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:10.278963 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495a063-44e3-4e40-98e5-573950667c9e None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2/users/57908f89930446f486457c86720dd372/roles/c1c0bce9f7a34def89550deb2e26113a` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:10.284001 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495a063-44e3-4e40-98e5-573950667c9e None admin] RBAC: Authorizing `identity:create_grant(project_id=8959cf36bec74b8ca22049270ee47cb2, user_id=57908f89930446f486457c86720dd372, role_id=c1c0bce9f7a34def89550deb2e26113a)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:10.285851 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495a063-44e3-4e40-98e5-573950667c9e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:10.304562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5d05e-88f2-48e0-8c38-fd2c33f7c7e6 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:10.357758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6d016-0750-48cc-a1c9-183fae1317a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:10.358431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6d016-0750-48cc-a1c9-183fae1317a5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:10.358713 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6d016-0750-48cc-a1c9-183fae1317a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:10.359070 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6d016-0750-48cc-a1c9-183fae1317a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:10.359813 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6d016-0750-48cc-a1c9-183fae1317a5 None admin] RBAC: Adding query filter params (role.id=c44d082d88894b888e94797acf7655b7, scope.project.id=8959cf36bec74b8ca22049270ee47cb2, user.id=fda9ca2d3fb54751b2eb0636906d0c8c) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:10.360127 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6d016-0750-48cc-a1c9-183fae1317a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:10.360627 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 82/162] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:17:09 2022] PUT /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2/users/57908f89930446f486457c86720dd372/roles/c1c0bce9f7a34def89550deb2e26113a => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:17:10.361679 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6d016-0750-48cc-a1c9-183fae1317a5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:10.361943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2cdf68-f3d2-425c-924e-cdf6234f38b5 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:10.362236 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2cdf68-f3d2-425c-924e-cdf6234f38b5 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:10.362515 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2cdf68-f3d2-425c-924e-cdf6234f38b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:10.374031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 81/163] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:17:10 2022] GET /identity/v3/role_assignments?user.id=fda9ca2d3fb54751b2eb0636906d0c8c&scope.project.id=8959cf36bec74b8ca22049270ee47cb2&role.id=c44d082d88894b888e94797acf7655b7 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:10.375303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6d016-0750-48cc-a1c9-183fae1317a5 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:10.409684 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-aa2cdf68-f3d2-425c-924e-cdf6234f38b5 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:10.411379 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-aa2cdf68-f3d2-425c-924e-cdf6234f38b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:10.541737 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 83/164] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:10.592103 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:10.597467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6d016-0750-48cc-a1c9-183fae1317a5 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:10.645969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae41e80-f972-49e3-a5a2-ad2691021bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:10.646982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae41e80-f972-49e3-a5a2-ad2691021bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:10.647429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae41e80-f972-49e3-a5a2-ad2691021bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:10.647870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae41e80-f972-49e3-a5a2-ad2691021bf4 None admin] PATH_INFO: `/v3/users/c847b76e4e6649e18dc342c5d998066b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:10.649278 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae41e80-f972-49e3-a5a2-ad2691021bf4 None admin] RBAC: Authorizing `identity:get_user(user_id=c847b76e4e6649e18dc342c5d998066b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:10.651201 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae41e80-f972-49e3-a5a2-ad2691021bf4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:10.654455 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 82/165] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:10 2022] GET /identity/v3/users/c847b76e4e6649e18dc342c5d998066b => generated 247 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:10.660079 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2cdf68-f3d2-425c-924e-cdf6234f38b5 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:10.673972 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae41e80-f972-49e3-a5a2-ad2691021bf4 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:10.840231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:10.882678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae41e80-f972-49e3-a5a2-ad2691021bf4 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:10.883749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2cdf68-f3d2-425c-924e-cdf6234f38b5 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:10.924788 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952ecea-6bce-4e77-9d07-e4f865fc073c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:10.925911 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952ecea-6bce-4e77-9d07-e4f865fc073c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:10.926189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952ecea-6bce-4e77-9d07-e4f865fc073c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:10.926460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952ecea-6bce-4e77-9d07-e4f865fc073c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:10.939233 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952ecea-6bce-4e77-9d07-e4f865fc073c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:10.944094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-6952ecea-6bce-4e77-9d07-e4f865fc073c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:10.944094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:10.944094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:10.944094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:10.944094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:10.944094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:10.944094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:10.944094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:10.944094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:10.944094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:10.944094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:10.944094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:10.944094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:10.944094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:10.944094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:10.944094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:10.944984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:10.944984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:10.944984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:10.944984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:10.944984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:10.944984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:10.944984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:10.944984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:10.944984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:10.944984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:10.944984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:10.944984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:10.944984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:10.944984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:10.944984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:10.944984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:10.946009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:10.946009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:10.946009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:10.946009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:10.946009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:10.946009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:10.946009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:10.946009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:10.946009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:10.946009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:10.946009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:10.946009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:10.954883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 83/166] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:17:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:17:10.960880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952ecea-6bce-4e77-9d07-e4f865fc073c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:10.962267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e3459-bfff-4bb1-99b5-d1e19f531316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:10.962879 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e3459-bfff-4bb1-99b5-d1e19f531316 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:10.963150 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e3459-bfff-4bb1-99b5-d1e19f531316 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:10.963420 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e3459-bfff-4bb1-99b5-d1e19f531316 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:11.020245 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:11.067292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e3459-bfff-4bb1-99b5-d1e19f531316 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:11.070252 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e3459-bfff-4bb1-99b5-d1e19f531316 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:11.178469 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:11.207845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952ecea-6bce-4e77-9d07-e4f865fc073c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:11.208068 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 84/167] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:11.215773 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e3459-bfff-4bb1-99b5-d1e19f531316 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:11.286610 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff6121-23e3-4cc8-a835-a0a14af0eec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:11.287235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff6121-23e3-4cc8-a835-a0a14af0eec8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:11.287514 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff6121-23e3-4cc8-a835-a0a14af0eec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:11.287788 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff6121-23e3-4cc8-a835-a0a14af0eec8 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:11.288571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff6121-23e3-4cc8-a835-a0a14af0eec8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:11.288943 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff6121-23e3-4cc8-a835-a0a14af0eec8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:11.290366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff6121-23e3-4cc8-a835-a0a14af0eec8 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:11.311184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 84/168] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:17:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:11.312122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff6121-23e3-4cc8-a835-a0a14af0eec8 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:11.347445 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e3459-bfff-4bb1-99b5-d1e19f531316 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:11.428368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd31e4-73c1-4e68-be8e-275d016301a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:11.428368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd31e4-73c1-4e68-be8e-275d016301a4 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:11.428368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd31e4-73c1-4e68-be8e-275d016301a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:11.428368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd31e4-73c1-4e68-be8e-275d016301a4 None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:11.429651 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd31e4-73c1-4e68-be8e-275d016301a4 None admin] RBAC: Authorizing `identity:get_project(project_id=8959cf36bec74b8ca22049270ee47cb2)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:11.431391 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd31e4-73c1-4e68-be8e-275d016301a4 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:11.433280 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 85/169] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:11 2022] GET /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:11.434373 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd31e4-73c1-4e68-be8e-275d016301a4 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:11.517484 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:11.537081 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:11.568485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff6121-23e3-4cc8-a835-a0a14af0eec8 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:11.635529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd31e4-73c1-4e68-be8e-275d016301a4 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:11.644773 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97dd0ba-3f06-4b58-b67f-b3b57c9dd5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:11.645741 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dd0ba-3f06-4b58-b67f-b3b57c9dd5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:11.646229 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dd0ba-3f06-4b58-b67f-b3b57c9dd5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:11.646720 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dd0ba-3f06-4b58-b67f-b3b57c9dd5b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:11.669747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97dd0ba-3f06-4b58-b67f-b3b57c9dd5b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:11.675890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-a97dd0ba-3f06-4b58-b67f-b3b57c9dd5b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:11.675890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:11.675890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:11.675890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:11.675890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:11.675890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:11.675890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:11.675890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:11.675890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:11.675890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:11.675890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:11.675890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:11.675890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:11.675890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:11.675890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:11.675890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:11.676792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:11.676792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:11.676792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:11.676792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:11.676792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:11.676792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:11.676792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:11.676792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:11.676792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:11.676792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:11.676792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:11.676792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:11.676792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:11.676792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:11.676792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:11.676792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:11.677634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:11.677634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:11.677634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:11.677634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:11.677634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:11.677634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:11.677634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:11.677634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:11.677634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:11.677634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:11.677634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:11.677634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:11.677634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 85/170] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:11.679176 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97dd0ba-3f06-4b58-b67f-b3b57c9dd5b6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:11.692785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18152450-15bf-4c99-8b38-733bd6cd7165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:11.693366 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18152450-15bf-4c99-8b38-733bd6cd7165 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:11.693636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18152450-15bf-4c99-8b38-733bd6cd7165 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:11.693905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18152450-15bf-4c99-8b38-733bd6cd7165 None admin] PATH_INFO: `/v3/users/fda9ca2d3fb54751b2eb0636906d0c8c` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:11.699187 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18152450-15bf-4c99-8b38-733bd6cd7165 None admin] RBAC: Authorizing `identity:get_user(user_id=fda9ca2d3fb54751b2eb0636906d0c8c)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:11.700638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18152450-15bf-4c99-8b38-733bd6cd7165 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:11.702423 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 86/171] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:11 2022] GET /identity/v3/users/fda9ca2d3fb54751b2eb0636906d0c8c => generated 275 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:11.708205 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18152450-15bf-4c99-8b38-733bd6cd7165 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:11.868609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97dd0ba-3f06-4b58-b67f-b3b57c9dd5b6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:11.884088 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18152450-15bf-4c99-8b38-733bd6cd7165 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:11.909624 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d0e77-0d5e-4716-914d-f5dc26688e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:11.911179 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d0e77-0d5e-4716-914d-f5dc26688e58 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:11.911653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d0e77-0d5e-4716-914d-f5dc26688e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:11.912258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d0e77-0d5e-4716-914d-f5dc26688e58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:11.913218 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d0e77-0d5e-4716-914d-f5dc26688e58 None admin] RBAC: Adding query filter params (role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.project.id=8959cf36bec74b8ca22049270ee47cb2, user.id=c847b76e4e6649e18dc342c5d998066b) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:11.913722 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d0e77-0d5e-4716-914d-f5dc26688e58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:11.915359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d0e77-0d5e-4716-914d-f5dc26688e58 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:11.915847 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:11.923650 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 86/172] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:17:11 2022] GET /identity/v3/role_assignments?user.id=c847b76e4e6649e18dc342c5d998066b&scope.project.id=8959cf36bec74b8ca22049270ee47cb2&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:11.925010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d0e77-0d5e-4716-914d-f5dc26688e58 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:11.946937 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:11.961085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669dfd83-5c25-44ba-a5d7-f65f9750dcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:11.961085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669dfd83-5c25-44ba-a5d7-f65f9750dcc7 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:11.961085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669dfd83-5c25-44ba-a5d7-f65f9750dcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:11.961085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669dfd83-5c25-44ba-a5d7-f65f9750dcc7 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:11.961682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669dfd83-5c25-44ba-a5d7-f65f9750dcc7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:11.961682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669dfd83-5c25-44ba-a5d7-f65f9750dcc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:11.961682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669dfd83-5c25-44ba-a5d7-f65f9750dcc7 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:11.980838 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 87/173] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:11.987016 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669dfd83-5c25-44ba-a5d7-f65f9750dcc7 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:12.115775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d0e77-0d5e-4716-914d-f5dc26688e58 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:12.181904 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193aa6ea-55f7-4043-ae32-243b2a6689ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:12.182924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193aa6ea-55f7-4043-ae32-243b2a6689ac None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:12.183397 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193aa6ea-55f7-4043-ae32-243b2a6689ac None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:12.183868 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193aa6ea-55f7-4043-ae32-243b2a6689ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:12.195275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669dfd83-5c25-44ba-a5d7-f65f9750dcc7 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:12.263403 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91459a-1eb8-430e-ba67-64ba5dbc4a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:12.265176 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91459a-1eb8-430e-ba67-64ba5dbc4a35 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:12.268272 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91459a-1eb8-430e-ba67-64ba5dbc4a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:12.268775 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91459a-1eb8-430e-ba67-64ba5dbc4a35 None admin] PATH_INFO: `/v3/users/fda9ca2d3fb54751b2eb0636906d0c8c` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:12.270170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91459a-1eb8-430e-ba67-64ba5dbc4a35 None admin] RBAC: Authorizing `identity:get_user(user_id=fda9ca2d3fb54751b2eb0636906d0c8c)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:12.271843 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91459a-1eb8-430e-ba67-64ba5dbc4a35 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:12.274637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 88/174] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:11 2022] GET /identity/v3/users/fda9ca2d3fb54751b2eb0636906d0c8c => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:12.277847 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414f877-edb4-44bb-9923-b0c447338424 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:12.279793 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414f877-edb4-44bb-9923-b0c447338424 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:12.280418 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414f877-edb4-44bb-9923-b0c447338424 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:12.282912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 89/175] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:12.285202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414f877-edb4-44bb-9923-b0c447338424 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:12.301699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193aa6ea-55f7-4043-ae32-243b2a6689ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:12.303503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193aa6ea-55f7-4043-ae32-243b2a6689ac None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:12.321959 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:12.375339 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:12.437807 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 87/176] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:12.439171 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98b88d-19a5-4f89-84a2-c3f1d646346f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:12.439468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98b88d-19a5-4f89-84a2-c3f1d646346f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:12.439748 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98b88d-19a5-4f89-84a2-c3f1d646346f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:12.463161 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4f98b88d-19a5-4f89-84a2-c3f1d646346f None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:12.464666 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4f98b88d-19a5-4f89-84a2-c3f1d646346f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:12.511911 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414f877-edb4-44bb-9923-b0c447338424 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:12.547959 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 88/177] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:12.549217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57095d8-6375-48d4-8405-982052145301 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:12.549494 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57095d8-6375-48d4-8405-982052145301 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:12.549767 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57095d8-6375-48d4-8405-982052145301 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:12.551240 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 89/178] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:12.551971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57095d8-6375-48d4-8405-982052145301 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:12.556470 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4569480-0491-4dd5-937c-259e39ef8689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:12.557354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4569480-0491-4dd5-937c-259e39ef8689 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:12.557893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4569480-0491-4dd5-937c-259e39ef8689 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:12.558646 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4569480-0491-4dd5-937c-259e39ef8689 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:12.602173 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:12.652121 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4569480-0491-4dd5-937c-259e39ef8689 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:12.654852 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4569480-0491-4dd5-937c-259e39ef8689 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:12.676015 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57095d8-6375-48d4-8405-982052145301 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:12.711508 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:12.723048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1349fe9-352f-416c-8342-9ec9eb135813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:12.724181 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1349fe9-352f-416c-8342-9ec9eb135813 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:12.724810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1349fe9-352f-416c-8342-9ec9eb135813 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:12.725294 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1349fe9-352f-416c-8342-9ec9eb135813 None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:12.727118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1349fe9-352f-416c-8342-9ec9eb135813 None admin] RBAC: Authorizing `identity:get_project(project_id=8959cf36bec74b8ca22049270ee47cb2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:12.728877 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1349fe9-352f-416c-8342-9ec9eb135813 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:12.731534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 90/179] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:12 2022] GET /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:12.733268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020b957-e391-44ce-89b5-f51f119c6fcb None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:12.734516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020b957-e391-44ce-89b5-f51f119c6fcb None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:12.734991 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020b957-e391-44ce-89b5-f51f119c6fcb None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:12.736547 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 91/180] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:12.738008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2a4f3-275a-4742-9e61-aeb965c74a9e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:12.738549 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2a4f3-275a-4742-9e61-aeb965c74a9e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:12.739592 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2a4f3-275a-4742-9e61-aeb965c74a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:12.763237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a7a2a4f3-275a-4742-9e61-aeb965c74a9e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:12.764695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a7a2a4f3-275a-4742-9e61-aeb965c74a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:12.770670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 90/181] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:12.771665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4569480-0491-4dd5-937c-259e39ef8689 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:12.868944 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 92/182] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:12.874476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657a715-b673-44a4-9817-13c3fa721b79 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:12.874895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657a715-b673-44a4-9817-13c3fa721b79 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:12.875176 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657a715-b673-44a4-9817-13c3fa721b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:12.894634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-0657a715-b673-44a4-9817-13c3fa721b79 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:12.895833 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0657a715-b673-44a4-9817-13c3fa721b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:12.897852 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4569480-0491-4dd5-937c-259e39ef8689 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:12.931909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:12.937482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9beee-963b-42d8-8acf-143ffd255043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:12.938352 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9beee-963b-42d8-8acf-143ffd255043 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:12.938880 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9beee-963b-42d8-8acf-143ffd255043 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:12.939363 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9beee-963b-42d8-8acf-143ffd255043 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:12.940335 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9beee-963b-42d8-8acf-143ffd255043 None admin] RBAC: Adding query filter params (role.id=e4309dcb11f74e10ba108b7b00264476, scope.project.id=8959cf36bec74b8ca22049270ee47cb2, user.id=fda9ca2d3fb54751b2eb0636906d0c8c) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:12.940866 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9beee-963b-42d8-8acf-143ffd255043 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:12.942545 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9beee-963b-42d8-8acf-143ffd255043 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:12.948841 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 91/183] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:17:12 2022] GET /identity/v3/role_assignments?user.id=fda9ca2d3fb54751b2eb0636906d0c8c&scope.project.id=8959cf36bec74b8ca22049270ee47cb2&role.id=e4309dcb11f74e10ba108b7b00264476 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:12.949971 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9beee-963b-42d8-8acf-143ffd255043 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:12.990473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 93/184] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:12.991378 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657a715-b673-44a4-9817-13c3fa721b79 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:13.005025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:13.110106 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9beee-963b-42d8-8acf-143ffd255043 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:13.145086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657a715-b673-44a4-9817-13c3fa721b79 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:13.183189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ebd69-ec02-44dc-bcd1-debf9b6d0f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:13.183967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ebd69-ec02-44dc-bcd1-debf9b6d0f30 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:13.184315 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ebd69-ec02-44dc-bcd1-debf9b6d0f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:13.184659 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ebd69-ec02-44dc-bcd1-debf9b6d0f30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:13.189494 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261a19d-86eb-4324-8bc8-e43ab8ebe75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:13.190154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261a19d-86eb-4324-8bc8-e43ab8ebe75d None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:13.190433 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261a19d-86eb-4324-8bc8-e43ab8ebe75d None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:13.190706 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261a19d-86eb-4324-8bc8-e43ab8ebe75d None admin] PATH_INFO: `/v3/projects/570be56d739e4e869a2dc14432b5ec6c` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:13.195049 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261a19d-86eb-4324-8bc8-e43ab8ebe75d None admin] RBAC: Authorizing `identity:get_project(project_id=570be56d739e4e869a2dc14432b5ec6c)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:13.196525 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261a19d-86eb-4324-8bc8-e43ab8ebe75d None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:13.197012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ebd69-ec02-44dc-bcd1-debf9b6d0f30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:13.198320 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 92/185] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:12 2022] GET /identity/v3/projects/570be56d739e4e869a2dc14432b5ec6c => generated 312 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:13.202992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-6c3ebd69-ec02-44dc-bcd1-debf9b6d0f30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:17:13.202992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:13.202992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:13.202992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:13.202992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:13.202992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:13.202992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:13.202992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:13.202992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:13.202992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:13.202992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:13.202992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:13.202992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:13.202992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:13.202992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:13.202992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:13.204342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:13.204342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:13.204342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:13.204342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:13.204342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:13.204342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:13.204342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:13.204342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:13.204342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:13.204342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:13.204342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:13.204342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:13.204342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:13.204342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:13.204342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:13.204342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:13.205217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:13.205217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:13.205217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:13.205217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:13.205217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:13.205217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:13.205217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:13.205217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:13.205217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:13.205217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:13.205217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:17:13.205217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:13.205217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 94/186] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:17:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:13.206696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261a19d-86eb-4324-8bc8-e43ab8ebe75d None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:13.211785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ebd69-ec02-44dc-bcd1-debf9b6d0f30 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:13.257569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:13.337090 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ebd69-ec02-44dc-bcd1-debf9b6d0f30 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:13.348197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:13.373714 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261a19d-86eb-4324-8bc8-e43ab8ebe75d None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:13.379459 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edab4d-87e8-40e3-a365-03eebdb0bf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:13.380532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edab4d-87e8-40e3-a365-03eebdb0bf0e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:13.380823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edab4d-87e8-40e3-a365-03eebdb0bf0e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:13.381153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edab4d-87e8-40e3-a365-03eebdb0bf0e None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:13.381840 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edab4d-87e8-40e3-a365-03eebdb0bf0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:13.382150 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edab4d-87e8-40e3-a365-03eebdb0bf0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:13.383932 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edab4d-87e8-40e3-a365-03eebdb0bf0e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:13.397489 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 95/187] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:13.398352 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edab4d-87e8-40e3-a365-03eebdb0bf0e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:13.423341 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c933733-9044-4f92-a8d2-93c317e3b5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:13.423961 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c933733-9044-4f92-a8d2-93c317e3b5fb None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:13.424231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c933733-9044-4f92-a8d2-93c317e3b5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:13.424561 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c933733-9044-4f92-a8d2-93c317e3b5fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:13.425293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c933733-9044-4f92-a8d2-93c317e3b5fb None admin] RBAC: Adding query filter params (role.id=c44d082d88894b888e94797acf7655b7, scope.project.id=570be56d739e4e869a2dc14432b5ec6c, user.id=fda9ca2d3fb54751b2eb0636906d0c8c) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:13.425947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c933733-9044-4f92-a8d2-93c317e3b5fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:13.427227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c933733-9044-4f92-a8d2-93c317e3b5fb None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:13.433613 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 93/188] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:17:13 2022] GET /identity/v3/role_assignments?user.id=fda9ca2d3fb54751b2eb0636906d0c8c&scope.project.id=570be56d739e4e869a2dc14432b5ec6c&role.id=c44d082d88894b888e94797acf7655b7 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:13.437500 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c933733-9044-4f92-a8d2-93c317e3b5fb None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:13.537643 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:13.549343 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edab4d-87e8-40e3-a365-03eebdb0bf0e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:13.580858 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c933733-9044-4f92-a8d2-93c317e3b5fb None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:13.600476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54cccd-7da4-4b80-bae9-d320b5aed43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:13.601177 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54cccd-7da4-4b80-bae9-d320b5aed43f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:13.601658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54cccd-7da4-4b80-bae9-d320b5aed43f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:13.601942 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54cccd-7da4-4b80-bae9-d320b5aed43f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:13.613834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54cccd-7da4-4b80-bae9-d320b5aed43f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:13.618895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-dd54cccd-7da4-4b80-bae9-d320b5aed43f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:13.618895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:13.618895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:13.618895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:13.618895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:13.618895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:13.618895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:13.618895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:13.618895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:13.618895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:13.618895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:13.618895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:13.618895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:13.618895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:13.618895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:13.618895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:13.619816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:13.619816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:13.619816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:13.619816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:13.619816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:13.619816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:13.619816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:13.619816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:13.619816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:13.619816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:13.619816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:13.619816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:13.619816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:13.619816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:13.619816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:13.619816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:13.620600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:13.620600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:13.620600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:13.620600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:13.620600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:13.620600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:13.620600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:13.620600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:13.620600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:13.620600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:13.620600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:13.620600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:13.620600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 96/189] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:17:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:13.621758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54cccd-7da4-4b80-bae9-d320b5aed43f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:13.627792 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a3c02-04e4-46a2-9e3c-7578f94142a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:13.628378 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a3c02-04e4-46a2-9e3c-7578f94142a4 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:13.628657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a3c02-04e4-46a2-9e3c-7578f94142a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:13.628932 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a3c02-04e4-46a2-9e3c-7578f94142a4 None admin] PATH_INFO: `/v3/users/0915d03dd4c7443e80b4e3b88ebc0597` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:13.645233 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8a3c02-04e4-46a2-9e3c-7578f94142a4 None admin] RBAC: Authorizing `identity:get_user(user_id=0915d03dd4c7443e80b4e3b88ebc0597)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:13.646735 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8a3c02-04e4-46a2-9e3c-7578f94142a4 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:13.649122 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 94/190] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:13 2022] GET /identity/v3/users/0915d03dd4c7443e80b4e3b88ebc0597 => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:13.654902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf480d7-fe04-4b51-9e18-86ef801af246 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:13.654902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf480d7-fe04-4b51-9e18-86ef801af246 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:13.654902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf480d7-fe04-4b51-9e18-86ef801af246 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:13.654902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 95/191] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:13.654902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf480d7-fe04-4b51-9e18-86ef801af246 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:13.656309 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:13.736811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54cccd-7da4-4b80-bae9-d320b5aed43f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:13.764589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf480d7-fe04-4b51-9e18-86ef801af246 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:13.769885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eadaa1-ece4-4050-92a8-72a7cab1ae53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:13.770506 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eadaa1-ece4-4050-92a8-72a7cab1ae53 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:13.770785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eadaa1-ece4-4050-92a8-72a7cab1ae53 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:13.771060 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eadaa1-ece4-4050-92a8-72a7cab1ae53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:13.772185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eadaa1-ece4-4050-92a8-72a7cab1ae53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:13.773658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eadaa1-ece4-4050-92a8-72a7cab1ae53 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:13.776635 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 97/192] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:17:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:13.776635 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eadaa1-ece4-4050-92a8-72a7cab1ae53 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:13.800123 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:13.801632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1b71b-0379-4dd4-9056-247d81a4aef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:13.802235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1b71b-0379-4dd4-9056-247d81a4aef6 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:13.802512 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1b71b-0379-4dd4-9056-247d81a4aef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:13.802780 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1b71b-0379-4dd4-9056-247d81a4aef6 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:13.803430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1b71b-0379-4dd4-9056-247d81a4aef6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:13.803727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1b71b-0379-4dd4-9056-247d81a4aef6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:13.805176 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1b71b-0379-4dd4-9056-247d81a4aef6 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:13.813700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 96/193] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:17:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:13.814848 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34baf8-4215-4ae4-9dca-214b34f10a4f None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:13.815128 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34baf8-4215-4ae4-9dca-214b34f10a4f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:13.815488 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34baf8-4215-4ae4-9dca-214b34f10a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:13.834180 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-eb34baf8-4215-4ae4-9dca-214b34f10a4f None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:13.835379 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-eb34baf8-4215-4ae4-9dca-214b34f10a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:13.883138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eadaa1-ece4-4050-92a8-72a7cab1ae53 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:13.893576 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:13.902008 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 97/194] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:13.902787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34baf8-4215-4ae4-9dca-214b34f10a4f None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:13.924613 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e2f3a-0feb-486e-9b53-e19f5bdf977d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:13.925590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e2f3a-0feb-486e-9b53-e19f5bdf977d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:13.926009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e2f3a-0feb-486e-9b53-e19f5bdf977d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:13.926755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e2f3a-0feb-486e-9b53-e19f5bdf977d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:13.997321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9e2f3a-0feb-486e-9b53-e19f5bdf977d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:13.998886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9e2f3a-0feb-486e-9b53-e19f5bdf977d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:14.008098 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34baf8-4215-4ae4-9dca-214b34f10a4f None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:14.019543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:14.044034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50656fdd-d9a8-4a2e-a5ad-5beff3766b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:14.044879 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50656fdd-d9a8-4a2e-a5ad-5beff3766b6d None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:14.045336 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50656fdd-d9a8-4a2e-a5ad-5beff3766b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:14.045612 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50656fdd-d9a8-4a2e-a5ad-5beff3766b6d None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:14.046834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50656fdd-d9a8-4a2e-a5ad-5beff3766b6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:14.048688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50656fdd-d9a8-4a2e-a5ad-5beff3766b6d None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:14.098561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 98/195] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:14.100482 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e2f3a-0feb-486e-9b53-e19f5bdf977d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:14.149733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 98/196] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 19 10:17:13 2022] POST /identity/v3/users => generated 297 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:17:14.152287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50656fdd-d9a8-4a2e-a5ad-5beff3766b6d None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:14.242525 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:14.251486 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e2f3a-0feb-486e-9b53-e19f5bdf977d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:14.285838 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:14.306695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f137af-be29-49c9-abb4-94ec6f57eb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:14.307425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f137af-be29-49c9-abb4-94ec6f57eb5e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:14.307701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f137af-be29-49c9-abb4-94ec6f57eb5e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:14.307980 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f137af-be29-49c9-abb4-94ec6f57eb5e None admin] PATH_INFO: `/v3/users/0915d03dd4c7443e80b4e3b88ebc0597` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:14.309347 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f137af-be29-49c9-abb4-94ec6f57eb5e None admin] RBAC: Authorizing `identity:get_user(user_id=0915d03dd4c7443e80b4e3b88ebc0597)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:14.311218 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f137af-be29-49c9-abb4-94ec6f57eb5e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:14.312840 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 99/197] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:14 2022] GET /identity/v3/users/0915d03dd4c7443e80b4e3b88ebc0597 => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:14.313710 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f137af-be29-49c9-abb4-94ec6f57eb5e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:14.334172 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50656fdd-d9a8-4a2e-a5ad-5beff3766b6d None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:14.375306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be11eba5-2ca1-4804-9aaa-af5d5ccafd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:14.376286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11eba5-2ca1-4804-9aaa-af5d5ccafd1c None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:14.377213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11eba5-2ca1-4804-9aaa-af5d5ccafd1c None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:14.377491 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11eba5-2ca1-4804-9aaa-af5d5ccafd1c None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:14.378646 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be11eba5-2ca1-4804-9aaa-af5d5ccafd1c None admin] RBAC: Authorizing `identity:get_project(project_id=a52b1cb46c36451498d5787d735e4f12)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:14.380782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be11eba5-2ca1-4804-9aaa-af5d5ccafd1c None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:14.381991 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 99/198] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:14 2022] GET /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:14.382549 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be11eba5-2ca1-4804-9aaa-af5d5ccafd1c None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:14.442036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f137af-be29-49c9-abb4-94ec6f57eb5e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:14.487857 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709589ca-5be3-4175-bce6-389fdb3f7de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:14.488469 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709589ca-5be3-4175-bce6-389fdb3f7de7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:14.488743 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709589ca-5be3-4175-bce6-389fdb3f7de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:14.489020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709589ca-5be3-4175-bce6-389fdb3f7de7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:14.502445 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709589ca-5be3-4175-bce6-389fdb3f7de7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:14.503327 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be11eba5-2ca1-4804-9aaa-af5d5ccafd1c None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:14.507873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-709589ca-5be3-4175-bce6-389fdb3f7de7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 10:17:14.507873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:14.507873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:14.507873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:14.507873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:14.507873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:14.507873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:14.507873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:14.507873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:14.507873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:14.507873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:14.507873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:14.507873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:14.507873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:14.507873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:14.507873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:14.508808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:14.508808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:14.508808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:14.508808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:14.508808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:14.508808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:14.508808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:14.508808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:14.508808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:14.508808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:14.508808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:14.508808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:14.508808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:14.508808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:14.508808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:14.508808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:14.510058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:14.510058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:14.510058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:14.510058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:14.510058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:14.510058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:14.510058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:14.510058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:14.510058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:14.510058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:14.510058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 10:17:14.510058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:14.510058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 100/199] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:14.514050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709589ca-5be3-4175-bce6-389fdb3f7de7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:14.523883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:14.543620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e53d5-ef26-4820-b90b-e5317415bd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:14.544245 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e53d5-ef26-4820-b90b-e5317415bd76 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:14.544527 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e53d5-ef26-4820-b90b-e5317415bd76 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:14.544801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e53d5-ef26-4820-b90b-e5317415bd76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:14.573736 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:14.633890 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e53d5-ef26-4820-b90b-e5317415bd76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:14.635378 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e53d5-ef26-4820-b90b-e5317415bd76 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:14.638903 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709589ca-5be3-4175-bce6-389fdb3f7de7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:14.679901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0eec1-d339-4476-9736-f473a7f98a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:14.681361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0eec1-d339-4476-9736-f473a7f98a23 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:14.681963 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0eec1-d339-4476-9736-f473a7f98a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:14.682442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0eec1-d339-4476-9736-f473a7f98a23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:14.683807 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0eec1-d339-4476-9736-f473a7f98a23 None admin] RBAC: Adding query filter params (role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.project.id=a52b1cb46c36451498d5787d735e4f12, user.id=0915d03dd4c7443e80b4e3b88ebc0597) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:14.684571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0eec1-d339-4476-9736-f473a7f98a23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:14.686421 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0eec1-d339-4476-9736-f473a7f98a23 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:14.693352 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 101/200] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:17:14 2022] GET /identity/v3/role_assignments?user.id=0915d03dd4c7443e80b4e3b88ebc0597&scope.project.id=a52b1cb46c36451498d5787d735e4f12&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:14.694500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0eec1-d339-4476-9736-f473a7f98a23 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:14.803419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 100/201] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:14.807381 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e53d5-ef26-4820-b90b-e5317415bd76 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:14.841807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:14.935837 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0eec1-d339-4476-9736-f473a7f98a23 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:14.955962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e53d5-ef26-4820-b90b-e5317415bd76 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:14.967907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:14.995274 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76544880-258b-412f-a250-991d5a2e1818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:14.995963 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76544880-258b-412f-a250-991d5a2e1818 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:14.996245 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76544880-258b-412f-a250-991d5a2e1818 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:14.996521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76544880-258b-412f-a250-991d5a2e1818 None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:14.997750 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76544880-258b-412f-a250-991d5a2e1818 None admin] RBAC: Authorizing `identity:get_project(project_id=a52b1cb46c36451498d5787d735e4f12)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:15.000437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76544880-258b-412f-a250-991d5a2e1818 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:15.003284 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 101/202] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:14 2022] GET /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:15.009076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76544880-258b-412f-a250-991d5a2e1818 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:15.023548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e13ba-5ea1-4c23-806d-b67fcdb09773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:15.024459 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e13ba-5ea1-4c23-806d-b67fcdb09773 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:15.024741 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e13ba-5ea1-4c23-806d-b67fcdb09773 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:15.025015 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e13ba-5ea1-4c23-806d-b67fcdb09773 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:15.025965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e13ba-5ea1-4c23-806d-b67fcdb09773 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:15.026272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e13ba-5ea1-4c23-806d-b67fcdb09773 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:15.027643 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e13ba-5ea1-4c23-806d-b67fcdb09773 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:15.054709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 102/203] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:15.062840 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e13ba-5ea1-4c23-806d-b67fcdb09773 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:15.169421 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:15.188171 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76544880-258b-412f-a250-991d5a2e1818 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:15.234498 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff740c-7929-4d88-a433-05582b925917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:15.235432 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff740c-7929-4d88-a433-05582b925917 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:15.235902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff740c-7929-4d88-a433-05582b925917 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:15.236382 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff740c-7929-4d88-a433-05582b925917 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:15.237482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff740c-7929-4d88-a433-05582b925917 None admin] RBAC: Adding query filter params (role.id=e4309dcb11f74e10ba108b7b00264476, scope.project.id=a52b1cb46c36451498d5787d735e4f12, user.id=0915d03dd4c7443e80b4e3b88ebc0597) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:15.237990 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff740c-7929-4d88-a433-05582b925917 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:15.239911 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff740c-7929-4d88-a433-05582b925917 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:15.247003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 102/204] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:17:15 2022] GET /identity/v3/role_assignments?user.id=0915d03dd4c7443e80b4e3b88ebc0597&scope.project.id=a52b1cb46c36451498d5787d735e4f12&role.id=e4309dcb11f74e10ba108b7b00264476 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:15.283590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e13ba-5ea1-4c23-806d-b67fcdb09773 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:15.339917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74217c0a-d99e-4643-821b-ea28f4a351d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:15.340557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74217c0a-d99e-4643-821b-ea28f4a351d2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:15.340838 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74217c0a-d99e-4643-821b-ea28f4a351d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:15.341146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74217c0a-d99e-4643-821b-ea28f4a351d2 None admin] PATH_INFO: `/v3/users/57908f89930446f486457c86720dd372` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:15.342323 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74217c0a-d99e-4643-821b-ea28f4a351d2 None admin] RBAC: Authorizing `identity:get_user(user_id=57908f89930446f486457c86720dd372)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:15.344341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74217c0a-d99e-4643-821b-ea28f4a351d2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:15.346313 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 103/205] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:15 2022] GET /identity/v3/users/57908f89930446f486457c86720dd372 => generated 289 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:15.352077 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff740c-7929-4d88-a433-05582b925917 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:15.495200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff740c-7929-4d88-a433-05582b925917 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:15.534218 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c5f74-0157-4dd0-9592-b19a05390997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:15.534837 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c5f74-0157-4dd0-9592-b19a05390997 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:15.535119 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c5f74-0157-4dd0-9592-b19a05390997 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:15.535396 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c5f74-0157-4dd0-9592-b19a05390997 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:15.558794 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:15.614146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c5f74-0157-4dd0-9592-b19a05390997 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:15.615830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c5f74-0157-4dd0-9592-b19a05390997 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:15.737618 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 103/206] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:15.743458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74217c0a-d99e-4643-821b-ea28f4a351d2 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:15.832608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:15.906551 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74217c0a-d99e-4643-821b-ea28f4a351d2 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:15.954655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce241387-33ad-4b7e-8dc6-8a97e886a26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:15.955293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce241387-33ad-4b7e-8dc6-8a97e886a26a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:15.955577 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce241387-33ad-4b7e-8dc6-8a97e886a26a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:15.955858 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce241387-33ad-4b7e-8dc6-8a97e886a26a None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:15.957013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce241387-33ad-4b7e-8dc6-8a97e886a26a None admin] RBAC: Authorizing `identity:get_project(project_id=8959cf36bec74b8ca22049270ee47cb2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:15.958533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce241387-33ad-4b7e-8dc6-8a97e886a26a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:15.960173 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 104/207] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:15 2022] GET /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:15.966455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c5f74-0157-4dd0-9592-b19a05390997 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:16.045881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:16.119382 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c5f74-0157-4dd0-9592-b19a05390997 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:16.172141 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50758f92-e776-4772-b8e3-5f2d52b3334b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:16.172907 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50758f92-e776-4772-b8e3-5f2d52b3334b None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:16.173209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50758f92-e776-4772-b8e3-5f2d52b3334b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:16.173581 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50758f92-e776-4772-b8e3-5f2d52b3334b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:16.175168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50758f92-e776-4772-b8e3-5f2d52b3334b None admin] RBAC: Adding query filter params (role.id=c1c0bce9f7a34def89550deb2e26113a, scope.project.id=8959cf36bec74b8ca22049270ee47cb2, user.id=57908f89930446f486457c86720dd372) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:16.175483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50758f92-e776-4772-b8e3-5f2d52b3334b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:16.176928 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50758f92-e776-4772-b8e3-5f2d52b3334b None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:16.183474 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 104/208] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:17:15 2022] GET /identity/v3/role_assignments?user.id=57908f89930446f486457c86720dd372&scope.project.id=8959cf36bec74b8ca22049270ee47cb2&role.id=c1c0bce9f7a34def89550deb2e26113a => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:16.806022 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e545ab-1bf6-4325-bf13-2909614d296a None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:16.806022 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e545ab-1bf6-4325-bf13-2909614d296a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:16.806022 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e545ab-1bf6-4325-bf13-2909614d296a None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:16.806022 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 105/209] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:16.813102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd36bde-ff59-4665-a85c-6e270b9ad3a1 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:16.813743 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd36bde-ff59-4665-a85c-6e270b9ad3a1 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:16.814255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd36bde-ff59-4665-a85c-6e270b9ad3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:16.834240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-abd36bde-ff59-4665-a85c-6e270b9ad3a1 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:16.835715 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-abd36bde-ff59-4665-a85c-6e270b9ad3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:16.914821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7003a1-2343-430a-8be0-dbf75b5e510a None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:16.915183 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7003a1-2343-430a-8be0-dbf75b5e510a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:16.915366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7003a1-2343-430a-8be0-dbf75b5e510a None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:16.916868 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 106/210] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:17:16.923069 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 105/211] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:17:16.926889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e0459-428e-4f52-984a-51adcbb265cb None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:16.927427 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e0459-428e-4f52-984a-51adcbb265cb None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:16.927899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e0459-428e-4f52-984a-51adcbb265cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:16.946945 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e94e0459-428e-4f52-984a-51adcbb265cb None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:16.948447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e94e0459-428e-4f52-984a-51adcbb265cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:17.023054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 107/212] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:17.155955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c879aa-7a06-4423-87a4-79747c70697a None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:17.156895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c879aa-7a06-4423-87a4-79747c70697a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:17.157423 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c879aa-7a06-4423-87a4-79747c70697a None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:17.159382 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 106/213] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:17.164625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063bd15-169a-4488-995c-58ed4a468f45 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:17.165155 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063bd15-169a-4488-995c-58ed4a468f45 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:17.165677 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063bd15-169a-4488-995c-58ed4a468f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:17.184770 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b063bd15-169a-4488-995c-58ed4a468f45 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:17.186265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b063bd15-169a-4488-995c-58ed4a468f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:17.212695 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c879aa-7a06-4423-87a4-79747c70697a None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:17.222647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:17.262258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 108/214] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:17.278595 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:17.320183 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063bd15-169a-4488-995c-58ed4a468f45 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:17.327633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c879aa-7a06-4423-87a4-79747c70697a None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:17.363350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edaa72f-177a-4852-b70b-c06aa8728a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:17.364330 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edaa72f-177a-4852-b70b-c06aa8728a90 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:17.364832 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edaa72f-177a-4852-b70b-c06aa8728a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:17.365347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edaa72f-177a-4852-b70b-c06aa8728a90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:17.376145 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edaa72f-177a-4852-b70b-c06aa8728a90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:17.381209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-5edaa72f-177a-4852-b70b-c06aa8728a90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:17.381209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:17.381209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:17.381209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:17.381209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:17.381209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:17.381209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:17.381209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:17.381209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:17.381209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:17.381209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:17.381209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:17.381209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:17.381209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:17.381209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:17.381209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:17.382222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:17.382222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:17.382222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:17.382222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:17.382222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:17.382222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:17.382222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:17.382222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:17.382222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:17.382222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:17.382222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:17.382222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:17.382222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:17.382222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:17.382222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:17.382222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:17.383026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:17.383026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:17.383026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:17.383026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:17.383026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:17.383026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:17.383026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:17.383026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:17.383026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:17.383026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:17.383026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:17.383026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:17.385311 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 107/215] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:17:17.390583 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edaa72f-177a-4852-b70b-c06aa8728a90 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:17.439534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063bd15-169a-4488-995c-58ed4a468f45 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:17.485386 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec41f4-ad62-443a-a290-330da37b3d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:17.486020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec41f4-ad62-443a-a290-330da37b3d07 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:17.486301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec41f4-ad62-443a-a290-330da37b3d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:17.486569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec41f4-ad62-443a-a290-330da37b3d07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:17.487694 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec41f4-ad62-443a-a290-330da37b3d07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:17.489125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec41f4-ad62-443a-a290-330da37b3d07 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:17.490688 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 109/216] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:17:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:17.496116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec41f4-ad62-443a-a290-330da37b3d07 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:17.534263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edaa72f-177a-4852-b70b-c06aa8728a90 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:17.537050 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:17.539870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:17.572533 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc601976-cc7f-4112-befc-ba95859dbbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:17.573175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc601976-cc7f-4112-befc-ba95859dbbd0 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:17.573449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc601976-cc7f-4112-befc-ba95859dbbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:17.573721 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc601976-cc7f-4112-befc-ba95859dbbd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:17.574379 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc601976-cc7f-4112-befc-ba95859dbbd0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:17.574677 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc601976-cc7f-4112-befc-ba95859dbbd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:17.576045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc601976-cc7f-4112-befc-ba95859dbbd0 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:17.589700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 108/217] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:17.589700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc601976-cc7f-4112-befc-ba95859dbbd0 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:17.603000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec41f4-ad62-443a-a290-330da37b3d07 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:17.638108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac244c-975f-450d-a6d3-b40326e1f275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:17.638712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac244c-975f-450d-a6d3-b40326e1f275 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:17.638994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac244c-975f-450d-a6d3-b40326e1f275 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:17.639267 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac244c-975f-450d-a6d3-b40326e1f275 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:17.639946 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac244c-975f-450d-a6d3-b40326e1f275 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:17.641398 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac244c-975f-450d-a6d3-b40326e1f275 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:17.696475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc601976-cc7f-4112-befc-ba95859dbbd0 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:17.711630 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 110/218] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 19 10:17:17 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:17:17.712724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac244c-975f-450d-a6d3-b40326e1f275 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:17.744527 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306c697-b095-4731-80dd-0dfa27a85957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:17.745411 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306c697-b095-4731-80dd-0dfa27a85957 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:17.745677 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306c697-b095-4731-80dd-0dfa27a85957 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:17.746078 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306c697-b095-4731-80dd-0dfa27a85957 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:17.766627 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306c697-b095-4731-80dd-0dfa27a85957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:17.775035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-c306c697-b095-4731-80dd-0dfa27a85957 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:17:17.775035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:17.775035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:17.775035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:17.775035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:17.775035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:17.775035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:17.775035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:17.775035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:17.775035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:17.775035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:17.775035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:17.775035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:17.775035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:17.775035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:17.775035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:17.776286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:17.776286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:17.776286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:17.776286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:17.776286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:17.776286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:17.776286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:17.776286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:17.776286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:17.776286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:17.776286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:17.776286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:17.776286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:17.776286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:17.776286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:17.776286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:17.777614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:17.777614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:17.777614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:17.777614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:17.777614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:17.777614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:17.777614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:17.777614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:17.777614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:17.777614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:17.777614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:17:17.777614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:17.777614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 109/219] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:17:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:17.784519 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306c697-b095-4731-80dd-0dfa27a85957 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:17.825597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:17.843396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:17.850879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac244c-975f-450d-a6d3-b40326e1f275 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:17.892670 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ab52a-94d5-47c4-bc59-d33c19423049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:17.893621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ab52a-94d5-47c4-bc59-d33c19423049 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:17.897085 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ab52a-94d5-47c4-bc59-d33c19423049 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:17.897085 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ab52a-94d5-47c4-bc59-d33c19423049 None admin] PATH_INFO: `/v3/users/8a2855f838794b7087831a6e4672187e` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:17.911710 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ab52a-94d5-47c4-bc59-d33c19423049 None admin] RBAC: Authorizing `identity:get_user(user_id=8a2855f838794b7087831a6e4672187e)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:17.913469 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ab52a-94d5-47c4-bc59-d33c19423049 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:17.915316 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 111/220] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:17 2022] GET /identity/v3/users/8a2855f838794b7087831a6e4672187e => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:17.916669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c229b5a-7d56-4b72-8c0d-7917f6ca0a44 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:17.917124 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c229b5a-7d56-4b72-8c0d-7917f6ca0a44 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:17.917578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c229b5a-7d56-4b72-8c0d-7917f6ca0a44 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:17.919036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 112/221] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:17.925195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58076fb-0400-402e-a84e-b051207b14cd None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:17.925598 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58076fb-0400-402e-a84e-b051207b14cd None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:17.925884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58076fb-0400-402e-a84e-b051207b14cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:17.931419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306c697-b095-4731-80dd-0dfa27a85957 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:17.946416 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c58076fb-0400-402e-a84e-b051207b14cd None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:17.947713 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c58076fb-0400-402e-a84e-b051207b14cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:17.971151 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57610646-1f63-411a-a0b4-85fed2655d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:17.971776 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57610646-1f63-411a-a0b4-85fed2655d8b None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:17.972054 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57610646-1f63-411a-a0b4-85fed2655d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:17.972329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57610646-1f63-411a-a0b4-85fed2655d8b None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:17.972994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57610646-1f63-411a-a0b4-85fed2655d8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:17.973335 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57610646-1f63-411a-a0b4-85fed2655d8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:17.974731 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57610646-1f63-411a-a0b4-85fed2655d8b None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:17.985181 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 110/222] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:17.985181 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57610646-1f63-411a-a0b4-85fed2655d8b None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:18.027427 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 113/223] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:17:18.028815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58076fb-0400-402e-a84e-b051207b14cd None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:18.097623 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:18.114468 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57610646-1f63-411a-a0b4-85fed2655d8b None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:18.129586 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:18.158817 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58076fb-0400-402e-a84e-b051207b14cd None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:18.162669 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9390be4-a84a-4c24-80ae-9f9093ad3cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:18.163284 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9390be4-a84a-4c24-80ae-9f9093ad3cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:18.163560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9390be4-a84a-4c24-80ae-9f9093ad3cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:18.163834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9390be4-a84a-4c24-80ae-9f9093ad3cd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:18.206450 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d0964-88f4-472f-9554-379249f0a647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:18.207036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d0964-88f4-472f-9554-379249f0a647 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:18.207300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d0964-88f4-472f-9554-379249f0a647 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:18.207583 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d0964-88f4-472f-9554-379249f0a647 None admin] PATH_INFO: `/v3/users/0915d03dd4c7443e80b4e3b88ebc0597` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:18.209290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d0964-88f4-472f-9554-379249f0a647 None admin] RBAC: Authorizing `identity:get_user(user_id=0915d03dd4c7443e80b4e3b88ebc0597)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:18.210776 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d0964-88f4-472f-9554-379249f0a647 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:18.213189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 114/224] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:18 2022] GET /identity/v3/users/0915d03dd4c7443e80b4e3b88ebc0597 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:18.219793 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d0964-88f4-472f-9554-379249f0a647 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:18.251614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9390be4-a84a-4c24-80ae-9f9093ad3cd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:18.254150 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9390be4-a84a-4c24-80ae-9f9093ad3cd3 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:18.331936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d0964-88f4-472f-9554-379249f0a647 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:18.356523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:18.368544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4355c-0a24-45c4-bfc3-603f7ca34db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:18.369210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4355c-0a24-45c4-bfc3-603f7ca34db3 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:18.373223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4355c-0a24-45c4-bfc3-603f7ca34db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:18.373358 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 111/225] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:18.373462 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4355c-0a24-45c4-bfc3-603f7ca34db3 None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:18.373462 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4355c-0a24-45c4-bfc3-603f7ca34db3 None admin] RBAC: Authorizing `identity:get_project(project_id=a52b1cb46c36451498d5787d735e4f12)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:18.373462 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4355c-0a24-45c4-bfc3-603f7ca34db3 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:18.375785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9390be4-a84a-4c24-80ae-9f9093ad3cd3 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:18.378423 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 115/226] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:18 2022] GET /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:18.380681 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4355c-0a24-45c4-bfc3-603f7ca34db3 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:18.400794 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:18.482009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9390be4-a84a-4c24-80ae-9f9093ad3cd3 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:18.485819 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4355c-0a24-45c4-bfc3-603f7ca34db3 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:18.518570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b5f68-d282-4c64-b0af-f3c091bf28de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:18.519478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b5f68-d282-4c64-b0af-f3c091bf28de None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:18.519905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b5f68-d282-4c64-b0af-f3c091bf28de None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:18.520332 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b5f68-d282-4c64-b0af-f3c091bf28de None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:18.521589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b5f68-d282-4c64-b0af-f3c091bf28de None admin] RBAC: Authorizing `identity:get_project(project_id=a52b1cb46c36451498d5787d735e4f12)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:18.522137 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c1cfa-732d-4790-834f-9bcf6cfb3f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:18.522966 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c1cfa-732d-4790-834f-9bcf6cfb3f9a None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:18.523080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b5f68-d282-4c64-b0af-f3c091bf28de None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:18.523486 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c1cfa-732d-4790-834f-9bcf6cfb3f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:18.523901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c1cfa-732d-4790-834f-9bcf6cfb3f9a None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12/users/0915d03dd4c7443e80b4e3b88ebc0597/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:18.524671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 112/227] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:18 2022] GET /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:18.526592 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c1cfa-732d-4790-834f-9bcf6cfb3f9a None admin] RBAC: Authorizing `identity:create_grant(project_id=a52b1cb46c36451498d5787d735e4f12, user_id=0915d03dd4c7443e80b4e3b88ebc0597, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:18.528189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c1cfa-732d-4790-834f-9bcf6cfb3f9a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:18.529711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b5f68-d282-4c64-b0af-f3c091bf28de None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:18.590067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:18.603744 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 116/228] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:17:18 2022] PUT /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12/users/0915d03dd4c7443e80b4e3b88ebc0597/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:17:18.606556 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c1cfa-732d-4790-834f-9bcf6cfb3f9a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:18.647603 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b5f68-d282-4c64-b0af-f3c091bf28de None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:18.697206 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0da7d3-065c-467f-99ba-626fc099d6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:18.698238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0da7d3-065c-467f-99ba-626fc099d6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:18.699862 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0da7d3-065c-467f-99ba-626fc099d6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:18.700428 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0da7d3-065c-467f-99ba-626fc099d6f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:18.725486 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0da7d3-065c-467f-99ba-626fc099d6f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:18.731546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-0a0da7d3-065c-467f-99ba-626fc099d6f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:18.731546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:18.731546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:18.731546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:18.731546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:18.731546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:18.731546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:18.731546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:18.731546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:18.731546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:18.731546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:18.731546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:18.731546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:18.731546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:18.731546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:18.731546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:18.732638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:18.732638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:18.732638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:18.732638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:18.732638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:18.732638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:18.732638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:18.732638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:18.732638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:18.732638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:18.732638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:18.732638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:18.732638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:18.732638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:18.732638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:18.732638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:18.733560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:18.733560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:18.733560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:18.733560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:18.733560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:18.733560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:18.733560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:18.733560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:18.733560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:18.733560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:18.733560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:18.733560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:18.736295 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 113/229] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:17:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:18.749455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0da7d3-065c-467f-99ba-626fc099d6f4 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:18.779181 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:18.803820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c1cfa-732d-4790-834f-9bcf6cfb3f9a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:18.849707 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95928a-af61-4369-bca1-6a8258aadf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:18.851020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95928a-af61-4369-bca1-6a8258aadf72 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:18.851445 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95928a-af61-4369-bca1-6a8258aadf72 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:18.851863 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95928a-af61-4369-bca1-6a8258aadf72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:18.852744 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb95928a-af61-4369-bca1-6a8258aadf72 None admin] RBAC: Adding query filter params (role.id=c44d082d88894b888e94797acf7655b7, scope.project.id=a52b1cb46c36451498d5787d735e4f12, user.id=8a2855f838794b7087831a6e4672187e) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:18.853229 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb95928a-af61-4369-bca1-6a8258aadf72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:18.854808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb95928a-af61-4369-bca1-6a8258aadf72 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:18.862488 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 117/230] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:17:18 2022] GET /identity/v3/role_assignments?user.id=8a2855f838794b7087831a6e4672187e&scope.project.id=a52b1cb46c36451498d5787d735e4f12&role.id=c44d082d88894b888e94797acf7655b7 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:18.891654 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0da7d3-065c-467f-99ba-626fc099d6f4 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:18.913173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:18.940039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e30077-337c-4714-a045-a30a17ee5825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:18.940680 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e30077-337c-4714-a045-a30a17ee5825 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:18.940961 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e30077-337c-4714-a045-a30a17ee5825 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:18.941265 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e30077-337c-4714-a045-a30a17ee5825 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:18.941954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e30077-337c-4714-a045-a30a17ee5825 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:18.942260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e30077-337c-4714-a045-a30a17ee5825 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:18.943669 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e30077-337c-4714-a045-a30a17ee5825 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:18.958101 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 114/231] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:17:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:18.964543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95928a-af61-4369-bca1-6a8258aadf72 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:19.076930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95928a-af61-4369-bca1-6a8258aadf72 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:19.122730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d18ea-224b-43bc-86a3-8bfcc2e59d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:19.123926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d18ea-224b-43bc-86a3-8bfcc2e59d1c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:19.124438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d18ea-224b-43bc-86a3-8bfcc2e59d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:19.124908 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d18ea-224b-43bc-86a3-8bfcc2e59d1c None admin] PATH_INFO: `/v3/users/0915d03dd4c7443e80b4e3b88ebc0597` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:19.126321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d18ea-224b-43bc-86a3-8bfcc2e59d1c None admin] RBAC: Authorizing `identity:get_user(user_id=0915d03dd4c7443e80b4e3b88ebc0597)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:19.127980 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d18ea-224b-43bc-86a3-8bfcc2e59d1c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:19.130119 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 118/232] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:18 2022] GET /identity/v3/users/0915d03dd4c7443e80b4e3b88ebc0597 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:19.135537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e30077-337c-4714-a045-a30a17ee5825 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:19.268017 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e30077-337c-4714-a045-a30a17ee5825 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:19.312179 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b7077-0413-4c9a-b390-20616d13520a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:19.312821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b7077-0413-4c9a-b390-20616d13520a None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:19.313123 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b7077-0413-4c9a-b390-20616d13520a None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:19.313446 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b7077-0413-4c9a-b390-20616d13520a None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:19.314672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b7077-0413-4c9a-b390-20616d13520a None admin] RBAC: Authorizing `identity:get_project(project_id=a52b1cb46c36451498d5787d735e4f12)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:19.316142 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b7077-0413-4c9a-b390-20616d13520a None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:19.318197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 115/233] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:19 2022] GET /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:19.323686 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d18ea-224b-43bc-86a3-8bfcc2e59d1c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:19.357160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:19.463854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d18ea-224b-43bc-86a3-8bfcc2e59d1c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:19.510210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78453ae3-7838-4b18-90bd-93861b190cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:19.510878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78453ae3-7838-4b18-90bd-93861b190cdb None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:19.511160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78453ae3-7838-4b18-90bd-93861b190cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:19.511440 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78453ae3-7838-4b18-90bd-93861b190cdb None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12/users/0915d03dd4c7443e80b4e3b88ebc0597/roles/e4309dcb11f74e10ba108b7b00264476` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:19.513668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78453ae3-7838-4b18-90bd-93861b190cdb None admin] RBAC: Authorizing `identity:create_grant(project_id=a52b1cb46c36451498d5787d735e4f12, user_id=0915d03dd4c7443e80b4e3b88ebc0597, role_id=e4309dcb11f74e10ba108b7b00264476)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:19.515249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78453ae3-7838-4b18-90bd-93861b190cdb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:19.632767 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 119/234] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:17:19 2022] PUT /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12/users/0915d03dd4c7443e80b4e3b88ebc0597/roles/e4309dcb11f74e10ba108b7b00264476 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:17:20.712394 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083cbf0-acf9-46b7-830a-f50ddb1d67ed None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:20.713386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083cbf0-acf9-46b7-830a-f50ddb1d67ed None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:20.713900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083cbf0-acf9-46b7-830a-f50ddb1d67ed None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:20.715781 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 116/235] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:20.722008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af22146-ecd6-4564-a3fb-470c93acb476 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:20.722885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af22146-ecd6-4564-a3fb-470c93acb476 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:20.723374 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af22146-ecd6-4564-a3fb-470c93acb476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:20.746361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6af22146-ecd6-4564-a3fb-470c93acb476 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:20.747866 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6af22146-ecd6-4564-a3fb-470c93acb476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:20.798436 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70f8fd-bc68-4b60-a1d7-89f5ac37188e None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:20.799077 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70f8fd-bc68-4b60-a1d7-89f5ac37188e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:20.799560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70f8fd-bc68-4b60-a1d7-89f5ac37188e None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:20.801386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 117/236] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:20.817891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919621b-23e6-4922-b909-7c7c11eb872a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:20.818518 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919621b-23e6-4922-b909-7c7c11eb872a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:20.819083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919621b-23e6-4922-b909-7c7c11eb872a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:20.838467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:20.845501 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 120/237] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:20.858073 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-e919621b-23e6-4922-b909-7c7c11eb872a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:20.860240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e919621b-23e6-4922-b909-7c7c11eb872a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:20.917083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:20.955543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 118/238] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:17:21.179807 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af22146-ecd6-4564-a3fb-470c93acb476 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:21.312726 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919621b-23e6-4922-b909-7c7c11eb872a None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:21.326568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af22146-ecd6-4564-a3fb-470c93acb476 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:21.372071 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ca74c-cb5b-454a-9084-84e5bc2805a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:21.373167 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ca74c-cb5b-454a-9084-84e5bc2805a1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:21.373663 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ca74c-cb5b-454a-9084-84e5bc2805a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:21.374170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ca74c-cb5b-454a-9084-84e5bc2805a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:21.375601 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ca74c-cb5b-454a-9084-84e5bc2805a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:21.377389 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ca74c-cb5b-454a-9084-84e5bc2805a1 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:21.384208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 121/239] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:17:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:21.385827 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9eaf3-f4a0-405e-bd0e-a1ecd06794c4 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:21.386364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9eaf3-f4a0-405e-bd0e-a1ecd06794c4 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:21.386832 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9eaf3-f4a0-405e-bd0e-a1ecd06794c4 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:21.388485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 122/240] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:21.397980 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9eaf3-f4a0-405e-bd0e-a1ecd06794c4 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:21.434736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919621b-23e6-4922-b909-7c7c11eb872a None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:21.508283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e673c-7228-4418-8dd5-5778afe61dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:21.508912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e673c-7228-4418-8dd5-5778afe61dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:21.509224 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e673c-7228-4418-8dd5-5778afe61dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:21.509497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e673c-7228-4418-8dd5-5778afe61dc9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:21.510489 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:21.520174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e673c-7228-4418-8dd5-5778afe61dc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:21.525281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-aa1e673c-7228-4418-8dd5-5778afe61dc9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 10:17:21.525281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:21.525281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:21.525281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:21.525281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:21.525281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:21.525281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:21.525281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:21.525281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:21.525281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:21.525281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:21.525281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:21.525281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:21.525281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:21.525281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:21.525281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:21.526177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:21.526177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:21.526177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:21.526177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:21.526177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:21.526177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:21.526177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:21.526177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:21.526177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:21.526177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:21.526177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:21.526177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:21.526177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:21.526177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:21.526177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:21.526177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:21.526994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:21.526994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:21.526994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:21.526994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:21.526994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:21.526994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:21.526994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:21.526994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:21.526994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:21.526994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:21.526994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 10:17:21.526994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:21.526994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 119/241] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:21.528187 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9b844-770c-4bbb-ba47-2d8c8e8bf068 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:21.528187 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9b844-770c-4bbb-ba47-2d8c8e8bf068 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:21.528187 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9b844-770c-4bbb-ba47-2d8c8e8bf068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:21.546413 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-b4c9b844-770c-4bbb-ba47-2d8c8e8bf068 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:21.547614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b4c9b844-770c-4bbb-ba47-2d8c8e8bf068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:21.551348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9eaf3-f4a0-405e-bd0e-a1ecd06794c4 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:21.577812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:21.587272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b27351-a357-4c4b-a2ed-a3b03f1db213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:21.589961 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b27351-a357-4c4b-a2ed-a3b03f1db213 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:21.589961 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b27351-a357-4c4b-a2ed-a3b03f1db213 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:21.589961 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b27351-a357-4c4b-a2ed-a3b03f1db213 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:21.589961 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b27351-a357-4c4b-a2ed-a3b03f1db213 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:21.590562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b27351-a357-4c4b-a2ed-a3b03f1db213 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:21.615198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 120/242] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:21.616506 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e18457-6efb-4a0c-ab3f-5d11831c7711 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:21.616809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e18457-6efb-4a0c-ab3f-5d11831c7711 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:21.617087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e18457-6efb-4a0c-ab3f-5d11831c7711 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:21.621131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 121/243] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:21.621131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e18457-6efb-4a0c-ab3f-5d11831c7711 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:21.689918 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 123/244] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 19 10:17:21 2022] POST /identity/v3/users => generated 293 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:17:21.691608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a087b-0acf-4458-b957-94bf3755b867 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:21.692110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a087b-0acf-4458-b957-94bf3755b867 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:21.692823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a087b-0acf-4458-b957-94bf3755b867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:21.716242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1a5a087b-0acf-4458-b957-94bf3755b867 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:21.718515 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1a5a087b-0acf-4458-b957-94bf3755b867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:21.744599 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e18457-6efb-4a0c-ab3f-5d11831c7711 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:21.821198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18dbc6-1d43-4df5-8265-2315358d7196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:21.821822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18dbc6-1d43-4df5-8265-2315358d7196 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:21.822103 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18dbc6-1d43-4df5-8265-2315358d7196 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:21.822380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18dbc6-1d43-4df5-8265-2315358d7196 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:21.823108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18dbc6-1d43-4df5-8265-2315358d7196 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:21.823415 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18dbc6-1d43-4df5-8265-2315358d7196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:21.824832 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18dbc6-1d43-4df5-8265-2315358d7196 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:21.841553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 122/245] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:21.845495 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 124/246] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:21.850775 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18dbc6-1d43-4df5-8265-2315358d7196 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:21.903433 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:21.972810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a087b-0acf-4458-b957-94bf3755b867 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:21.985844 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18dbc6-1d43-4df5-8265-2315358d7196 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:22.025528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e68a86-4f99-41da-82ec-2683dfe5840d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:22.026165 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e68a86-4f99-41da-82ec-2683dfe5840d None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:22.026447 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e68a86-4f99-41da-82ec-2683dfe5840d None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:22.026725 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e68a86-4f99-41da-82ec-2683dfe5840d None admin] PATH_INFO: `/v3/users/a4884d1729914f048141ea8e9576ea49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:22.033200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:22.041960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e68a86-4f99-41da-82ec-2683dfe5840d None admin] RBAC: Authorizing `identity:get_user(user_id=a4884d1729914f048141ea8e9576ea49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:22.043702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e68a86-4f99-41da-82ec-2683dfe5840d None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:22.046438 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 123/247] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:21 2022] GET /identity/v3/users/a4884d1729914f048141ea8e9576ea49 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:22.058003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e68a86-4f99-41da-82ec-2683dfe5840d None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:22.087636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a087b-0acf-4458-b957-94bf3755b867 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:22.129924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae979acc-8a6e-42e2-9dd4-2c3dc355bb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:22.130568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae979acc-8a6e-42e2-9dd4-2c3dc355bb55 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:22.130849 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae979acc-8a6e-42e2-9dd4-2c3dc355bb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:22.131127 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae979acc-8a6e-42e2-9dd4-2c3dc355bb55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:22.147826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae979acc-8a6e-42e2-9dd4-2c3dc355bb55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:22.154523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-ae979acc-8a6e-42e2-9dd4-2c3dc355bb55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:17:22.154523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:22.154523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:22.154523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:22.154523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:22.154523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:22.154523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:22.154523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:22.154523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:22.154523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:22.154523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:22.154523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:22.154523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:22.154523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:22.154523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:22.154523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:22.156265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:22.156265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:22.156265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:22.156265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:22.156265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:22.156265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:22.156265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:22.156265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:22.156265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:22.156265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:22.156265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:22.156265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:22.156265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:22.156265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:22.156265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:22.156265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:22.157341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:22.157341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:22.157341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:22.157341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:22.157341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:22.157341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:22.157341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:22.157341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:22.157341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:22.157341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:22.157341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:17:22.157341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:22.160596 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 125/248] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:17:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:17:22.165530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae979acc-8a6e-42e2-9dd4-2c3dc355bb55 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:22.189081 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:22.214503 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e68a86-4f99-41da-82ec-2683dfe5840d None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:22.266262 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea9a61-b263-4566-8e8c-44752ba886e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:22.266781 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea9a61-b263-4566-8e8c-44752ba886e7 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:22.267047 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea9a61-b263-4566-8e8c-44752ba886e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:22.267909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea9a61-b263-4566-8e8c-44752ba886e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:22.288422 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae979acc-8a6e-42e2-9dd4-2c3dc355bb55 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:22.305264 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:22.332132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a44a48-6615-4718-871d-80b8cde2c5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:22.332132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a44a48-6615-4718-871d-80b8cde2c5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:22.332132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a44a48-6615-4718-871d-80b8cde2c5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:22.332478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a44a48-6615-4718-871d-80b8cde2c5e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:22.334172 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a44a48-6615-4718-871d-80b8cde2c5e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:22.335385 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a44a48-6615-4718-871d-80b8cde2c5e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:22.336890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a44a48-6615-4718-871d-80b8cde2c5e0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:22.345767 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 126/249] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:22.346605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a44a48-6615-4718-871d-80b8cde2c5e0 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:22.361587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea9a61-b263-4566-8e8c-44752ba886e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:22.363091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea9a61-b263-4566-8e8c-44752ba886e7 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:22.474524 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a44a48-6615-4718-871d-80b8cde2c5e0 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:22.486056 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:22.508424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 124/250] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:22.511623 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa585c-ef12-40e2-99fa-1790e881c3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:22.512276 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa585c-ef12-40e2-99fa-1790e881c3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:22.512556 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa585c-ef12-40e2-99fa-1790e881c3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:22.513219 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa585c-ef12-40e2-99fa-1790e881c3b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:22.515359 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a4639-6584-43ed-b02e-8f60cc7c0268 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:22.516130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a4639-6584-43ed-b02e-8f60cc7c0268 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:22.516487 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a4639-6584-43ed-b02e-8f60cc7c0268 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:22.518381 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 125/251] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:22.518923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a4639-6584-43ed-b02e-8f60cc7c0268 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:22.536931 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa585c-ef12-40e2-99fa-1790e881c3b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:22.543616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-3caa585c-ef12-40e2-99fa-1790e881c3b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:22.543616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:22.543616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:22.543616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:22.543616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:22.543616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:22.543616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:22.543616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:22.543616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:22.543616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:22.543616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:22.543616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:22.543616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:22.543616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:22.543616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:22.543616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:22.544499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:22.544499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:22.544499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:22.544499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:22.544499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:22.544499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:22.544499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:22.544499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:22.544499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:22.544499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:22.544499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:22.544499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:22.544499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:22.544499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:22.544499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:22.544499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:22.545314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:22.545314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:22.545314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:22.545314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:22.545314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:22.545314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:22.545314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:22.545314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:22.545314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:22.545314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:22.545314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:22.545314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:22.546941 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 127/252] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:22.546941 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa585c-ef12-40e2-99fa-1790e881c3b8 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:22.592073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:22.645450 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a4639-6584-43ed-b02e-8f60cc7c0268 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:22.669529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa585c-ef12-40e2-99fa-1790e881c3b8 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:22.697097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5195e0a-22a8-497e-9540-be9d18beec81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:22.698067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5195e0a-22a8-497e-9540-be9d18beec81 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:22.698589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5195e0a-22a8-497e-9540-be9d18beec81 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:22.699004 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5195e0a-22a8-497e-9540-be9d18beec81 None admin] PATH_INFO: `/v3/users/0915d03dd4c7443e80b4e3b88ebc0597` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:22.700274 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5195e0a-22a8-497e-9540-be9d18beec81 None admin] RBAC: Authorizing `identity:get_user(user_id=0915d03dd4c7443e80b4e3b88ebc0597)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:22.702678 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5195e0a-22a8-497e-9540-be9d18beec81 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:22.704740 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 126/253] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:22 2022] GET /identity/v3/users/0915d03dd4c7443e80b4e3b88ebc0597 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:22.706156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb3dc9-2cab-4714-b227-de199a04cda2 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:22.706583 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb3dc9-2cab-4714-b227-de199a04cda2 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:22.706924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4633a1-ece8-45db-b3c2-6ed807fa2b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:22.707189 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb3dc9-2cab-4714-b227-de199a04cda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:22.707558 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4633a1-ece8-45db-b3c2-6ed807fa2b7a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:22.707837 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4633a1-ece8-45db-b3c2-6ed807fa2b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:22.708112 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4633a1-ece8-45db-b3c2-6ed807fa2b7a None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:22.709335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4633a1-ece8-45db-b3c2-6ed807fa2b7a None admin] RBAC: Authorizing `identity:get_project(project_id=a52b1cb46c36451498d5787d735e4f12)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:22.710881 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4633a1-ece8-45db-b3c2-6ed807fa2b7a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:22.712947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 128/254] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:22 2022] GET /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:22.713751 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4633a1-ece8-45db-b3c2-6ed807fa2b7a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:22.729556 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-d9fb3dc9-2cab-4714-b227-de199a04cda2 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:22.731180 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d9fb3dc9-2cab-4714-b227-de199a04cda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:22.787547 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:22.808285 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 127/255] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:22.809509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb3dc9-2cab-4714-b227-de199a04cda2 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:22.826813 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4633a1-ece8-45db-b3c2-6ed807fa2b7a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:22.838405 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:22.891032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6d5ac-cbd3-4d2b-a23b-2d154c2dbd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:22.891660 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6d5ac-cbd3-4d2b-a23b-2d154c2dbd26 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:22.891917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6d5ac-cbd3-4d2b-a23b-2d154c2dbd26 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:22.892196 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6d5ac-cbd3-4d2b-a23b-2d154c2dbd26 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:22.892901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6d5ac-cbd3-4d2b-a23b-2d154c2dbd26 None admin] RBAC: Adding query filter params (name=member) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:22.898819 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6d5ac-cbd3-4d2b-a23b-2d154c2dbd26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:22.900501 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6d5ac-cbd3-4d2b-a23b-2d154c2dbd26 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:22.917177 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 129/256] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:22.917951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6d5ac-cbd3-4d2b-a23b-2d154c2dbd26 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:22.928631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb3dc9-2cab-4714-b227-de199a04cda2 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:22.976687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11f9da-d238-446a-824f-7373b8479822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:22.977710 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11f9da-d238-446a-824f-7373b8479822 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:22.978218 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11f9da-d238-446a-824f-7373b8479822 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:22.978689 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11f9da-d238-446a-824f-7373b8479822 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:23.036025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:23.054126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11f9da-d238-446a-824f-7373b8479822 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:23.055930 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11f9da-d238-446a-824f-7373b8479822 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:23.113897 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6d5ac-cbd3-4d2b-a23b-2d154c2dbd26 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:23.152687 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04071b-cfb6-4eb1-88be-2fdd1b16c119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:23.153336 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04071b-cfb6-4eb1-88be-2fdd1b16c119 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:23.153614 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04071b-cfb6-4eb1-88be-2fdd1b16c119 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:23.153940 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04071b-cfb6-4eb1-88be-2fdd1b16c119 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:23.154625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04071b-cfb6-4eb1-88be-2fdd1b16c119 None admin] RBAC: Adding query filter params (role.id=c1c0bce9f7a34def89550deb2e26113a, scope.project.id=a52b1cb46c36451498d5787d735e4f12, user.id=a4884d1729914f048141ea8e9576ea49) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:23.154965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04071b-cfb6-4eb1-88be-2fdd1b16c119 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:23.156420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04071b-cfb6-4eb1-88be-2fdd1b16c119 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:23.162842 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 130/257] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:17:22 2022] GET /identity/v3/role_assignments?user.id=a4884d1729914f048141ea8e9576ea49&scope.project.id=a52b1cb46c36451498d5787d735e4f12&role.id=c1c0bce9f7a34def89550deb2e26113a => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:23.163990 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04071b-cfb6-4eb1-88be-2fdd1b16c119 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:23.183287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 128/258] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:23.190112 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11f9da-d238-446a-824f-7373b8479822 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:23.201446 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:23.277101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04071b-cfb6-4eb1-88be-2fdd1b16c119 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:23.313335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d4b0c-b277-468b-aa48-32bc90f48b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:23.313896 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d4b0c-b277-468b-aa48-32bc90f48b72 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:23.314176 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d4b0c-b277-468b-aa48-32bc90f48b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:23.314451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d4b0c-b277-468b-aa48-32bc90f48b72 None admin] PATH_INFO: `/v3/users/8a2855f838794b7087831a6e4672187e` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:23.315675 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d4b0c-b277-468b-aa48-32bc90f48b72 None admin] RBAC: Authorizing `identity:get_user(user_id=8a2855f838794b7087831a6e4672187e)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:23.317234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d4b0c-b277-468b-aa48-32bc90f48b72 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:23.319002 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 131/259] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:23 2022] GET /identity/v3/users/8a2855f838794b7087831a6e4672187e => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:23.319950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d4b0c-b277-468b-aa48-32bc90f48b72 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:23.355473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:23.384625 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11f9da-d238-446a-824f-7373b8479822 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:23.433297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af777474-1ad2-40dc-961e-46a510967d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:23.433958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af777474-1ad2-40dc-961e-46a510967d04 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:23.434236 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af777474-1ad2-40dc-961e-46a510967d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:23.434594 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af777474-1ad2-40dc-961e-46a510967d04 None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:23.436000 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af777474-1ad2-40dc-961e-46a510967d04 None admin] RBAC: Authorizing `identity:get_project(project_id=a52b1cb46c36451498d5787d735e4f12)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:23.438462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af777474-1ad2-40dc-961e-46a510967d04 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:23.441567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 129/260] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:23 2022] GET /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:23.442364 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af777474-1ad2-40dc-961e-46a510967d04 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:23.481501 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:23.487223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d4b0c-b277-468b-aa48-32bc90f48b72 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:23.575665 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f2731-a6a5-4159-8a7e-6084acf34e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:23.576314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f2731-a6a5-4159-8a7e-6084acf34e1a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:23.576593 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f2731-a6a5-4159-8a7e-6084acf34e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:23.576917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f2731-a6a5-4159-8a7e-6084acf34e1a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:23.587062 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af777474-1ad2-40dc-961e-46a510967d04 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:23.604250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f2731-a6a5-4159-8a7e-6084acf34e1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:23.614674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-461f2731-a6a5-4159-8a7e-6084acf34e1a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:23.614674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:23.614674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:23.614674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:23.614674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:23.614674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:23.614674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:23.614674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:23.614674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:23.614674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:23.614674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:23.614674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:23.614674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:23.614674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:23.614674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:23.614674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:23.615618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:23.615618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:23.615618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:23.615618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:23.615618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:23.615618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:23.615618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:23.615618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:23.615618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:23.615618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:23.615618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:23.615618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:23.615618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:23.615618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:23.615618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:23.615618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:23.616433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:23.616433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:23.616433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:23.616433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:23.616433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:23.616433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:23.616433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:23.616433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:23.616433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:23.616433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:23.616433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:23.616433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:23.624622 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 132/261] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:17:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:23.626988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f2731-a6a5-4159-8a7e-6084acf34e1a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:23.654984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e15f6f-37af-42a4-9e2e-1992b336ac6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:23.656124 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e15f6f-37af-42a4-9e2e-1992b336ac6b None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:23.656400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e15f6f-37af-42a4-9e2e-1992b336ac6b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:23.656672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e15f6f-37af-42a4-9e2e-1992b336ac6b None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:23.657943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e15f6f-37af-42a4-9e2e-1992b336ac6b None admin] RBAC: Authorizing `identity:get_project(project_id=a52b1cb46c36451498d5787d735e4f12)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:23.659395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e15f6f-37af-42a4-9e2e-1992b336ac6b None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:23.667901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 130/262] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:23 2022] GET /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:23.669693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e15f6f-37af-42a4-9e2e-1992b336ac6b None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:23.713534 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:23.830412 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f2731-a6a5-4159-8a7e-6084acf34e1a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:23.864854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ece83-c2e5-4401-8bcf-8ed23a5f093d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:23.865489 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ece83-c2e5-4401-8bcf-8ed23a5f093d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:23.865769 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ece83-c2e5-4401-8bcf-8ed23a5f093d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:23.866043 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ece83-c2e5-4401-8bcf-8ed23a5f093d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:23.866770 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ece83-c2e5-4401-8bcf-8ed23a5f093d None admin] RBAC: Adding query filter params (role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.project.id=a52b1cb46c36451498d5787d735e4f12, user.id=0915d03dd4c7443e80b4e3b88ebc0597) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:23.867080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ece83-c2e5-4401-8bcf-8ed23a5f093d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:23.868527 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ece83-c2e5-4401-8bcf-8ed23a5f093d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:23.870031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e15f6f-37af-42a4-9e2e-1992b336ac6b None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:23.881206 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 133/263] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:17:23 2022] GET /identity/v3/role_assignments?user.id=0915d03dd4c7443e80b4e3b88ebc0597&scope.project.id=a52b1cb46c36451498d5787d735e4f12&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:23.882149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ece83-c2e5-4401-8bcf-8ed23a5f093d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:23.927200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:23.953821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc2ba9-2967-429d-8fa5-b606558d1a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:23.954453 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc2ba9-2967-429d-8fa5-b606558d1a92 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:23.954728 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc2ba9-2967-429d-8fa5-b606558d1a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:23.954998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc2ba9-2967-429d-8fa5-b606558d1a92 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:23.955803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc2ba9-2967-429d-8fa5-b606558d1a92 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:23.956105 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc2ba9-2967-429d-8fa5-b606558d1a92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:23.957522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc2ba9-2967-429d-8fa5-b606558d1a92 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:23.976172 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 131/264] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:17:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:23.985834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc2ba9-2967-429d-8fa5-b606558d1a92 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:24.082572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ece83-c2e5-4401-8bcf-8ed23a5f093d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:24.140885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a8556-4cb7-45dc-ae9a-037b17399564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:24.141615 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a8556-4cb7-45dc-ae9a-037b17399564 None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:24.141895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a8556-4cb7-45dc-ae9a-037b17399564 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:24.142168 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a8556-4cb7-45dc-ae9a-037b17399564 None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12/users/8a2855f838794b7087831a6e4672187e/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:24.144416 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a8556-4cb7-45dc-ae9a-037b17399564 None admin] RBAC: Authorizing `identity:create_grant(project_id=a52b1cb46c36451498d5787d735e4f12, user_id=8a2855f838794b7087831a6e4672187e, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:24.145917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a8556-4cb7-45dc-ae9a-037b17399564 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:24.156749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:24.177218 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc2ba9-2967-429d-8fa5-b606558d1a92 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:24.219970 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e9826-930d-4d99-8cd3-8eff893aa30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:24.220610 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e9826-930d-4d99-8cd3-8eff893aa30e None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:24.220892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e9826-930d-4d99-8cd3-8eff893aa30e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:24.221197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e9826-930d-4d99-8cd3-8eff893aa30e None admin] PATH_INFO: `/v3/users/0915d03dd4c7443e80b4e3b88ebc0597` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:24.222341 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e9826-930d-4d99-8cd3-8eff893aa30e None admin] RBAC: Authorizing `identity:get_user(user_id=0915d03dd4c7443e80b4e3b88ebc0597)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:24.223792 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e9826-930d-4d99-8cd3-8eff893aa30e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:24.229645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 132/265] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:23 2022] GET /identity/v3/users/0915d03dd4c7443e80b4e3b88ebc0597 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:24.236498 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 134/266] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:17:23 2022] PUT /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12/users/8a2855f838794b7087831a6e4672187e/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:17:24.244498 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e9826-930d-4d99-8cd3-8eff893aa30e None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:24.383515 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e9826-930d-4d99-8cd3-8eff893aa30e None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:24.429305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09c1d3-5edd-413b-9ff5-a7480fcc88fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:24.429937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09c1d3-5edd-413b-9ff5-a7480fcc88fe None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:24.430216 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09c1d3-5edd-413b-9ff5-a7480fcc88fe None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:24.430488 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09c1d3-5edd-413b-9ff5-a7480fcc88fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:24.454658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:24.515733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a09c1d3-5edd-413b-9ff5-a7480fcc88fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:24.517286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a09c1d3-5edd-413b-9ff5-a7480fcc88fe None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:24.656147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 133/267] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:24.661783 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a8556-4cb7-45dc-ae9a-037b17399564 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:24.708555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ffb1c-62e9-4af7-96a8-a5ff557de869 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:24.708888 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ffb1c-62e9-4af7-96a8-a5ff557de869 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:24.709142 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ffb1c-62e9-4af7-96a8-a5ff557de869 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:24.710866 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 134/268] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:17:24.718107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0cc0e-e6bf-4373-bbd9-f13b4eabb1ab None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:24.718419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0cc0e-e6bf-4373-bbd9-f13b4eabb1ab None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:24.718659 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0cc0e-e6bf-4373-bbd9-f13b4eabb1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:24.740407 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-86d0cc0e-e6bf-4373-bbd9-f13b4eabb1ab None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:24.741665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-86d0cc0e-e6bf-4373-bbd9-f13b4eabb1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:24.762424 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:24.791830 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a8556-4cb7-45dc-ae9a-037b17399564 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:24.828044 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:24.833685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c66864-ab41-439f-9ec7-4d2721bdd994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:24.834587 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c66864-ab41-439f-9ec7-4d2721bdd994 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:24.835077 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c66864-ab41-439f-9ec7-4d2721bdd994 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:24.835553 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c66864-ab41-439f-9ec7-4d2721bdd994 None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:24.837704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c66864-ab41-439f-9ec7-4d2721bdd994 None admin] RBAC: Authorizing `identity:get_project(project_id=a52b1cb46c36451498d5787d735e4f12)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:24.839397 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c66864-ab41-439f-9ec7-4d2721bdd994 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:24.842881 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 135/269] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:24 2022] GET /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:24.844311 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958cbb5-b3a0-4ad9-823f-9a97324811bb None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:24.844805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958cbb5-b3a0-4ad9-823f-9a97324811bb None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:24.845309 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958cbb5-b3a0-4ad9-823f-9a97324811bb None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:24.846773 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 136/270] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:24.848274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958cbb5-b3a0-4ad9-823f-9a97324811bb None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:24.855252 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 135/271] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:24.856892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0e3b6-33c3-4214-960e-a69ed6b4d6ca None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:24.857216 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0e3b6-33c3-4214-960e-a69ed6b4d6ca None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:24.857489 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0e3b6-33c3-4214-960e-a69ed6b4d6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:24.881526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-23e0e3b6-33c3-4214-960e-a69ed6b4d6ca None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:24.883458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-23e0e3b6-33c3-4214-960e-a69ed6b4d6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:25.002072 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 136/272] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:25.014674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958cbb5-b3a0-4ad9-823f-9a97324811bb None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:25.051407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd61a1-f178-430a-92e5-74143c5885d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:25.052267 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd61a1-f178-430a-92e5-74143c5885d9 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:25.052744 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd61a1-f178-430a-92e5-74143c5885d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:25.053227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd61a1-f178-430a-92e5-74143c5885d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:25.054142 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd61a1-f178-430a-92e5-74143c5885d9 None admin] RBAC: Adding query filter params (role.id=e4309dcb11f74e10ba108b7b00264476, scope.project.id=a52b1cb46c36451498d5787d735e4f12, user.id=0915d03dd4c7443e80b4e3b88ebc0597) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:25.054609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd61a1-f178-430a-92e5-74143c5885d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:25.056238 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd61a1-f178-430a-92e5-74143c5885d9 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:25.069244 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 137/273] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:17:24 2022] GET /identity/v3/role_assignments?user.id=0915d03dd4c7443e80b4e3b88ebc0597&scope.project.id=a52b1cb46c36451498d5787d735e4f12&role.id=e4309dcb11f74e10ba108b7b00264476 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:25.196672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0e3b6-33c3-4214-960e-a69ed6b4d6ca None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:25.319670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0e3b6-33c3-4214-960e-a69ed6b4d6ca None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:25.333513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:25.343346 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd61a1-f178-430a-92e5-74143c5885d9 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:25.365085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644737d1-f9cb-436f-b6b6-4c8964425e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:25.366184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:25.366783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644737d1-f9cb-436f-b6b6-4c8964425e4e None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:25.367279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644737d1-f9cb-436f-b6b6-4c8964425e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:25.367924 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644737d1-f9cb-436f-b6b6-4c8964425e4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:25.370909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644737d1-f9cb-436f-b6b6-4c8964425e4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:25.372882 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644737d1-f9cb-436f-b6b6-4c8964425e4e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:25.374826 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 137/274] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:17:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:25.380511 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644737d1-f9cb-436f-b6b6-4c8964425e4e None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:25.474573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd61a1-f178-430a-92e5-74143c5885d9 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:25.500803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644737d1-f9cb-436f-b6b6-4c8964425e4e None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:25.514977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03a947-25e3-44cd-aaf0-3c2da10aa5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:25.516196 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03a947-25e3-44cd-aaf0-3c2da10aa5cb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:25.517114 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03a947-25e3-44cd-aaf0-3c2da10aa5cb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:25.517768 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03a947-25e3-44cd-aaf0-3c2da10aa5cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:25.530778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03a947-25e3-44cd-aaf0-3c2da10aa5cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:25.538846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-3d03a947-25e3-44cd-aaf0-3c2da10aa5cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:25.538846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:25.538846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:25.538846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:25.538846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:25.538846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:25.538846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:25.538846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:25.538846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:25.538846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:25.538846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:25.538846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:25.538846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:25.538846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:25.538846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:25.538846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:25.539810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:25.539810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:25.539810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:25.539810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:25.539810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:25.539810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:25.539810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:25.539810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:25.539810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:25.539810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:25.539810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:25.539810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:25.539810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:25.539810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:25.539810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:25.539810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:25.540625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:25.540625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:25.540625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:25.540625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:25.540625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:25.540625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:25.540625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:25.540625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:25.540625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:25.540625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:25.540625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:25.540625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:25.542641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 138/275] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:25.549635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f171ee-7ea1-44a8-b816-f0ac640fae1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:25.550279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f171ee-7ea1-44a8-b816-f0ac640fae1f None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:25.550572 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f171ee-7ea1-44a8-b816-f0ac640fae1f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:25.550675 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03a947-25e3-44cd-aaf0-3c2da10aa5cb None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:25.550874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f171ee-7ea1-44a8-b816-f0ac640fae1f None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:25.551615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f171ee-7ea1-44a8-b816-f0ac640fae1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:25.553276 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f171ee-7ea1-44a8-b816-f0ac640fae1f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:25.631481 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:25.655299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 138/276] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 19 10:17:25 2022] POST /identity/v3/users => generated 293 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:17:25.657247 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03a947-25e3-44cd-aaf0-3c2da10aa5cb None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:25.701212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e25da2-d719-4185-87df-85a7c62d179f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:25.702642 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e25da2-d719-4185-87df-85a7c62d179f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:25.703064 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e25da2-d719-4185-87df-85a7c62d179f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:25.703481 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e25da2-d719-4185-87df-85a7c62d179f None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:25.704345 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e25da2-d719-4185-87df-85a7c62d179f None admin] RBAC: Adding query filter params (name=member) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:25.704807 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e25da2-d719-4185-87df-85a7c62d179f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:25.706505 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e25da2-d719-4185-87df-85a7c62d179f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:25.721029 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 139/277] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:25.723951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f171ee-7ea1-44a8-b816-f0ac640fae1f None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:25.763088 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:25.852960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f171ee-7ea1-44a8-b816-f0ac640fae1f None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:25.889628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d3a66-fdbd-4148-84eb-59c2f81cf22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:25.890240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d3a66-fdbd-4148-84eb-59c2f81cf22a None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:25.890460 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d3a66-fdbd-4148-84eb-59c2f81cf22a None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:25.890912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d3a66-fdbd-4148-84eb-59c2f81cf22a None admin] PATH_INFO: `/v3/users/e7f9dab65285481b9dd18023a350a6cf` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:25.909873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d3a66-fdbd-4148-84eb-59c2f81cf22a None admin] RBAC: Authorizing `identity:get_user(user_id=e7f9dab65285481b9dd18023a350a6cf)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:25.911390 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d3a66-fdbd-4148-84eb-59c2f81cf22a None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:25.913559 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 139/278] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:25 2022] GET /identity/v3/users/e7f9dab65285481b9dd18023a350a6cf => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:25.919678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e25da2-d719-4185-87df-85a7c62d179f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:26.019873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc04a5f-48d7-4a1b-91f0-536948f13891 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:26.020511 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc04a5f-48d7-4a1b-91f0-536948f13891 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:26.020981 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc04a5f-48d7-4a1b-91f0-536948f13891 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:26.022562 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 140/279] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:26.028181 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3b85a-2ba9-454b-9ba2-017b4adb436f None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:26.028716 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3b85a-2ba9-454b-9ba2-017b4adb436f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:26.029256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3b85a-2ba9-454b-9ba2-017b4adb436f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:26.040931 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e25da2-d719-4185-87df-85a7c62d179f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:26.049024 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-f4d3b85a-2ba9-454b-9ba2-017b4adb436f None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:26.050666 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-f4d3b85a-2ba9-454b-9ba2-017b4adb436f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:26.090072 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c6a8b-8894-4b00-8861-1f0b9344ecd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:26.090678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c6a8b-8894-4b00-8861-1f0b9344ecd7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:26.090973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c6a8b-8894-4b00-8861-1f0b9344ecd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:26.091228 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c6a8b-8894-4b00-8861-1f0b9344ecd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:26.091959 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c6a8b-8894-4b00-8861-1f0b9344ecd7 None admin] RBAC: Adding query filter params (role.id=c44d082d88894b888e94797acf7655b7, user.id=e7f9dab65285481b9dd18023a350a6cf, scope.system=all) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:26.092808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c6a8b-8894-4b00-8861-1f0b9344ecd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:26.094399 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c6a8b-8894-4b00-8861-1f0b9344ecd7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:26.098234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:26.101163 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 140/280] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Sep 19 10:17:25 2022] GET /identity/v3/role_assignments?user.id=e7f9dab65285481b9dd18023a350a6cf&scope.system=all&role.id=c44d082d88894b888e94797acf7655b7 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:26.154840 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 141/281] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:17:26.483595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c6a8b-8894-4b00-8861-1f0b9344ecd7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:26.614973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c6a8b-8894-4b00-8861-1f0b9344ecd7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:26.660604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07a8c0-818c-425a-b693-4d59b5fe27e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:26.661259 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07a8c0-818c-425a-b693-4d59b5fe27e1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:26.661542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07a8c0-818c-425a-b693-4d59b5fe27e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:26.661818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07a8c0-818c-425a-b693-4d59b5fe27e1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:26.674040 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd07a8c0-818c-425a-b693-4d59b5fe27e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:26.679444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-fd07a8c0-818c-425a-b693-4d59b5fe27e1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 10:17:26.679444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:26.679444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:26.679444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:26.679444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:26.679444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:26.679444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:26.679444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:26.679444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:26.679444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:26.679444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:26.679444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:26.679444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:26.679444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:26.679444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:26.679444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:26.680359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:26.680359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:26.680359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:26.680359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:26.680359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:26.680359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:26.680359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:26.680359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:26.680359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:26.680359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:26.680359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:26.680359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:26.680359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:26.680359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:26.680359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:26.680359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:26.681612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:26.681612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:26.681612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:26.681612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:26.681612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:26.681612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:26.681612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:26.681612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:26.681612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:26.681612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:26.681612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 10:17:26.681612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:26.681612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 141/282] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:26.698632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3b85a-2ba9-454b-9ba2-017b4adb436f None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:26.739843 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:26.858329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3b85a-2ba9-454b-9ba2-017b4adb436f None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:26.913846 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7cf9e-cb8e-44d6-8d97-6c0c74b5c72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:26.915243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7cf9e-cb8e-44d6-8d97-6c0c74b5c72b None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:26.915730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7cf9e-cb8e-44d6-8d97-6c0c74b5c72b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:26.916289 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7cf9e-cb8e-44d6-8d97-6c0c74b5c72b None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:26.917197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7cf9e-cb8e-44d6-8d97-6c0c74b5c72b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:26.917736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7cf9e-cb8e-44d6-8d97-6c0c74b5c72b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:26.919333 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7cf9e-cb8e-44d6-8d97-6c0c74b5c72b None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:26.932610 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 142/283] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:26.939149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07a8c0-818c-425a-b693-4d59b5fe27e1 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:26.976338 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96462c-4783-4b23-b36c-f6c8b1c86c4e None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:26.977001 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96462c-4783-4b23-b36c-f6c8b1c86c4e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:26.977516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96462c-4783-4b23-b36c-f6c8b1c86c4e None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:26.979143 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 143/284] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:17:26.987689 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de17f72-c2fc-45e3-aaf8-668ae67e4f65 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:26.988313 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de17f72-c2fc-45e3-aaf8-668ae67e4f65 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:26.988822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de17f72-c2fc-45e3-aaf8-668ae67e4f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:27.015568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-9de17f72-c2fc-45e3-aaf8-668ae67e4f65 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:27.017182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-9de17f72-c2fc-45e3-aaf8-668ae67e4f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:27.020830 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:27.098261 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07a8c0-818c-425a-b693-4d59b5fe27e1 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:27.138171 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 144/285] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:17:27.153883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573e094-54f0-4390-8a49-c3711a79843e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:27.155443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573e094-54f0-4390-8a49-c3711a79843e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:27.155927 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573e094-54f0-4390-8a49-c3711a79843e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:27.156492 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573e094-54f0-4390-8a49-c3711a79843e None admin] PATH_INFO: `/v3/users/a4884d1729914f048141ea8e9576ea49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:27.158022 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b573e094-54f0-4390-8a49-c3711a79843e None admin] RBAC: Authorizing `identity:get_user(user_id=a4884d1729914f048141ea8e9576ea49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:27.159826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b573e094-54f0-4390-8a49-c3711a79843e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:27.166573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 142/286] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:26 2022] GET /identity/v3/users/a4884d1729914f048141ea8e9576ea49 => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:27.168325 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de17f72-c2fc-45e3-aaf8-668ae67e4f65 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:27.190233 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:27.307257 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de17f72-c2fc-45e3-aaf8-668ae67e4f65 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:27.341108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa29964-0e72-43b1-970d-c77f0140d540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:27.341736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa29964-0e72-43b1-970d-c77f0140d540 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:27.342011 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa29964-0e72-43b1-970d-c77f0140d540 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:27.342906 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa29964-0e72-43b1-970d-c77f0140d540 None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:27.344014 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa29964-0e72-43b1-970d-c77f0140d540 None admin] RBAC: Authorizing `identity:get_project(project_id=a52b1cb46c36451498d5787d735e4f12)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:27.345543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa29964-0e72-43b1-970d-c77f0140d540 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:27.347492 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 145/287] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:27 2022] GET /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:27.354364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573e094-54f0-4390-8a49-c3711a79843e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:27.498756 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573e094-54f0-4390-8a49-c3711a79843e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:27.512417 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa29964-0e72-43b1-970d-c77f0140d540 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:27.541517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:27.557520 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b11e68-f21e-445e-9436-360b2509f9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:27.558413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b11e68-f21e-445e-9436-360b2509f9ba None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:27.558863 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b11e68-f21e-445e-9436-360b2509f9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:27.559293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b11e68-f21e-445e-9436-360b2509f9ba None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12/users/a4884d1729914f048141ea8e9576ea49/roles/c1c0bce9f7a34def89550deb2e26113a` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:27.562186 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b11e68-f21e-445e-9436-360b2509f9ba None admin] RBAC: Authorizing `identity:create_grant(project_id=a52b1cb46c36451498d5787d735e4f12, user_id=a4884d1729914f048141ea8e9576ea49, role_id=c1c0bce9f7a34def89550deb2e26113a)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:27.563886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b11e68-f21e-445e-9436-360b2509f9ba None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:27.607200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:27.626472 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa29964-0e72-43b1-970d-c77f0140d540 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:27.641175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 143/288] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:17:27 2022] PUT /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12/users/a4884d1729914f048141ea8e9576ea49/roles/c1c0bce9f7a34def89550deb2e26113a => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:17:27.664738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea4e39-0602-467e-8b9f-c91d5b95b848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:27.665451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea4e39-0602-467e-8b9f-c91d5b95b848 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:27.665730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea4e39-0602-467e-8b9f-c91d5b95b848 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:27.666092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea4e39-0602-467e-8b9f-c91d5b95b848 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:27.683663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea4e39-0602-467e-8b9f-c91d5b95b848 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:27.699312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-99ea4e39-0602-467e-8b9f-c91d5b95b848 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:27.699312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:27.699312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:27.699312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:27.699312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:27.699312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:27.699312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:27.699312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:27.699312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:27.699312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:27.699312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:27.699312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:27.699312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:27.699312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:27.699312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:27.699312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:27.700290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:27.700290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:27.700290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:27.700290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:27.700290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:27.700290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:27.700290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:27.700290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:27.700290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:27.700290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:27.700290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:27.700290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:27.700290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:27.700290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:27.700290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:27.700290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:27.701560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:27.701560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:27.701560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:27.701560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:27.701560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:27.701560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:27.701560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:27.701560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:27.701560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:27.701560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:27.701560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:27.701560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:27.701560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 146/289] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:17:27.707144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b11e68-f21e-445e-9436-360b2509f9ba None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:27.866039 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b11e68-f21e-445e-9436-360b2509f9ba None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:27.910977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce18699-2cac-4cae-8adc-5291dbca787b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:27.911951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce18699-2cac-4cae-8adc-5291dbca787b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:27.912430 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce18699-2cac-4cae-8adc-5291dbca787b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:27.912922 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce18699-2cac-4cae-8adc-5291dbca787b None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:27.913811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce18699-2cac-4cae-8adc-5291dbca787b None admin] RBAC: Adding query filter params (name=member) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:27.914323 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce18699-2cac-4cae-8adc-5291dbca787b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:27.915882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce18699-2cac-4cae-8adc-5291dbca787b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:27.925946 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 144/290] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:27.931889 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea4e39-0602-467e-8b9f-c91d5b95b848 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:28.041827 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea4e39-0602-467e-8b9f-c91d5b95b848 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:28.084855 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661b12e-3997-4336-a804-18e55e873b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:28.085830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661b12e-3997-4336-a804-18e55e873b25 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:28.086304 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661b12e-3997-4336-a804-18e55e873b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:28.086794 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661b12e-3997-4336-a804-18e55e873b25 None admin] PATH_INFO: `/v3/users/8a2855f838794b7087831a6e4672187e` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:28.088128 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661b12e-3997-4336-a804-18e55e873b25 None admin] RBAC: Authorizing `identity:get_user(user_id=8a2855f838794b7087831a6e4672187e)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:28.089797 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661b12e-3997-4336-a804-18e55e873b25 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:28.095841 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 147/291] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:27 2022] GET /identity/v3/users/8a2855f838794b7087831a6e4672187e => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:28.097150 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce18699-2cac-4cae-8adc-5291dbca787b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:28.156045 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:28.225730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce18699-2cac-4cae-8adc-5291dbca787b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:28.280680 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64d3aa-8928-4396-ba80-e705c03485a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:28.281316 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64d3aa-8928-4396-ba80-e705c03485a7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:28.281595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64d3aa-8928-4396-ba80-e705c03485a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:28.281869 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64d3aa-8928-4396-ba80-e705c03485a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:28.362479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64d3aa-8928-4396-ba80-e705c03485a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:28.363951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64d3aa-8928-4396-ba80-e705c03485a7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:28.422273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:28.480159 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 145/292] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:28.486118 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661b12e-3997-4336-a804-18e55e873b25 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:28.501831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:28.607679 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661b12e-3997-4336-a804-18e55e873b25 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:28.615554 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f02d3e-097c-402c-813a-68f954a45786 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:28.615810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f02d3e-097c-402c-813a-68f954a45786 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:28.616043 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f02d3e-097c-402c-813a-68f954a45786 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:28.617575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 146/293] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:17:28.624452 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2b4a6-e2f7-43ce-bfb8-3291757fdf20 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:28.624728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2b4a6-e2f7-43ce-bfb8-3291757fdf20 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:28.625003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2b4a6-e2f7-43ce-bfb8-3291757fdf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:28.651975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-38d2b4a6-e2f7-43ce-bfb8-3291757fdf20 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:28.653195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-38d2b4a6-e2f7-43ce-bfb8-3291757fdf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:28.681722 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f5523-b179-489a-a329-49f2300fcf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:28.682365 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f5523-b179-489a-a329-49f2300fcf20 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:28.682643 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f5523-b179-489a-a329-49f2300fcf20 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:28.682915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f5523-b179-489a-a329-49f2300fcf20 None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:28.684486 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f5523-b179-489a-a329-49f2300fcf20 None admin] RBAC: Authorizing `identity:get_project(project_id=a52b1cb46c36451498d5787d735e4f12)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:28.686478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f5523-b179-489a-a329-49f2300fcf20 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:28.688739 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 148/294] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:28 2022] GET /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:28.694797 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f5523-b179-489a-a329-49f2300fcf20 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:28.749887 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 147/295] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:28.751145 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e37e04-21d8-4f56-a208-4c9f799e2c0d None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:28.751435 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e37e04-21d8-4f56-a208-4c9f799e2c0d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:28.751710 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e37e04-21d8-4f56-a208-4c9f799e2c0d None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:28.757381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 148/296] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:28.758548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13882f7-8a7a-4801-bfc0-415fd97dc8b5 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:28.758824 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13882f7-8a7a-4801-bfc0-415fd97dc8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:28.759092 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13882f7-8a7a-4801-bfc0-415fd97dc8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:28.794221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:28.806115 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b13882f7-8a7a-4801-bfc0-415fd97dc8b5 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:28.807294 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b13882f7-8a7a-4801-bfc0-415fd97dc8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:28.824994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f5523-b179-489a-a329-49f2300fcf20 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:28.865112 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f83c5-6031-48df-8075-6f8077c4be0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:28.865811 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f83c5-6031-48df-8075-6f8077c4be0a None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:28.866031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f83c5-6031-48df-8075-6f8077c4be0a None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:28.866304 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f83c5-6031-48df-8075-6f8077c4be0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:28.867140 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f83c5-6031-48df-8075-6f8077c4be0a None admin] RBAC: Adding query filter params (role.id=c44d082d88894b888e94797acf7655b7, scope.project.id=a52b1cb46c36451498d5787d735e4f12, user.id=8a2855f838794b7087831a6e4672187e) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:28.867449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f83c5-6031-48df-8075-6f8077c4be0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:28.868891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f83c5-6031-48df-8075-6f8077c4be0a None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:28.881220 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 149/297] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:17:28 2022] GET /identity/v3/role_assignments?user.id=8a2855f838794b7087831a6e4672187e&scope.project.id=a52b1cb46c36451498d5787d735e4f12&role.id=c44d082d88894b888e94797acf7655b7 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:28.931463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 149/298] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:29.035623 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f31b7-6cd6-442a-a68a-b5d7568bd230 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:29.036262 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f31b7-6cd6-442a-a68a-b5d7568bd230 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:29.036262 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f31b7-6cd6-442a-a68a-b5d7568bd230 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:29.037839 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 150/299] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:29.043385 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070e052-b175-496f-a10f-cf1ea3213dd1 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:29.043685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070e052-b175-496f-a10f-cf1ea3213dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:29.043930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070e052-b175-496f-a10f-cf1ea3213dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:29.066099 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-3070e052-b175-496f-a10f-cf1ea3213dd1 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:29.067290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3070e052-b175-496f-a10f-cf1ea3213dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:29.070115 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:29.148179 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 150/300] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:29.154122 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f31b7-6cd6-442a-a68a-b5d7568bd230 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:29.265670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f31b7-6cd6-442a-a68a-b5d7568bd230 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:29.293834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070e052-b175-496f-a10f-cf1ea3213dd1 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:29.314347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccabd8b-b712-490f-a17f-4f051363e959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:29.315320 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccabd8b-b712-490f-a17f-4f051363e959 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:29.315830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccabd8b-b712-490f-a17f-4f051363e959 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:29.316313 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccabd8b-b712-490f-a17f-4f051363e959 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:29.332821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccabd8b-b712-490f-a17f-4f051363e959 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:29.338582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-3ccabd8b-b712-490f-a17f-4f051363e959 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 10:17:29.338582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:29.338582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:29.338582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:29.338582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:29.338582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:29.338582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:29.338582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:29.338582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:29.338582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:29.338582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:29.338582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:29.338582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:29.338582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:29.338582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:29.338582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:29.339504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:29.339504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:29.339504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:29.339504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:29.339504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:29.339504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:29.339504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:29.339504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:29.339504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:29.339504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:29.339504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:29.339504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:29.339504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:29.339504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:29.339504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:29.339504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:29.340331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:29.340331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:29.340331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:29.340331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:29.340331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:29.340331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:29.340331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:29.340331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:29.340331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:29.340331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:29.340331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 10:17:29.340331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:29.342770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 151/301] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:29.349487 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccabd8b-b712-490f-a17f-4f051363e959 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:29.353487 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:29.423389 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070e052-b175-496f-a10f-cf1ea3213dd1 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:29.468411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc903191-26f8-41ef-9da5-7e8563cc810f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:29.470127 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc903191-26f8-41ef-9da5-7e8563cc810f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:29.470851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc903191-26f8-41ef-9da5-7e8563cc810f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:29.471287 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc903191-26f8-41ef-9da5-7e8563cc810f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:29.472825 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc903191-26f8-41ef-9da5-7e8563cc810f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:29.474451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc903191-26f8-41ef-9da5-7e8563cc810f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:29.475403 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccabd8b-b712-490f-a17f-4f051363e959 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:29.476741 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 151/302] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:17:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:29.483699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc903191-26f8-41ef-9da5-7e8563cc810f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:29.496506 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:29.512331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509f1f9-ebb9-45d6-8c62-8bfa47d0d627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:29.512943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509f1f9-ebb9-45d6-8c62-8bfa47d0d627 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:29.513282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509f1f9-ebb9-45d6-8c62-8bfa47d0d627 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:29.513543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509f1f9-ebb9-45d6-8c62-8bfa47d0d627 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:29.514707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509f1f9-ebb9-45d6-8c62-8bfa47d0d627 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:29.515123 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509f1f9-ebb9-45d6-8c62-8bfa47d0d627 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:29.516416 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509f1f9-ebb9-45d6-8c62-8bfa47d0d627 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:29.530138 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 152/303] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:29.530138 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509f1f9-ebb9-45d6-8c62-8bfa47d0d627 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:29.596174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:29.599946 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc903191-26f8-41ef-9da5-7e8563cc810f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:29.634043 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f760b0-0d17-4d0b-b8e7-9509d987b6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:29.634671 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f760b0-0d17-4d0b-b8e7-9509d987b6fe None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:29.634950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f760b0-0d17-4d0b-b8e7-9509d987b6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:29.635225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f760b0-0d17-4d0b-b8e7-9509d987b6fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:29.644107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509f1f9-ebb9-45d6-8c62-8bfa47d0d627 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:29.646039 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f760b0-0d17-4d0b-b8e7-9509d987b6fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:29.651621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-e5f760b0-0d17-4d0b-b8e7-9509d987b6fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:29.651621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:29.651621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:29.651621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:29.651621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:29.651621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:29.651621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:29.651621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:29.651621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:29.651621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:29.651621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:29.651621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:29.651621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:29.651621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:29.651621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:29.651621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:29.652525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:29.652525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:29.652525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:29.652525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:29.652525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:29.652525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:29.652525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:29.652525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:29.652525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:29.652525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:29.652525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:29.652525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:29.652525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:29.652525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:29.652525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:29.652525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:29.657815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:29.657815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:29.657815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:29.657815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:29.657815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:29.657815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:29.657815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:29.657815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:29.657815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:29.657815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:29.657815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:29.657815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:29.657815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 152/304] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:29.657815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f760b0-0d17-4d0b-b8e7-9509d987b6fe None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:29.681260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f1b32-2ebe-4606-8b55-76862ae41078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:29.681826 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f1b32-2ebe-4606-8b55-76862ae41078 None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:29.682105 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f1b32-2ebe-4606-8b55-76862ae41078 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:29.682373 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f1b32-2ebe-4606-8b55-76862ae41078 None admin] PATH_INFO: `/v3/groups` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:29.683009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f1b32-2ebe-4606-8b55-76862ae41078 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:29.684436 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f1b32-2ebe-4606-8b55-76862ae41078 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:29.728725 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:29.760480 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f760b0-0d17-4d0b-b8e7-9509d987b6fe None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:29.772568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 153/305] 10.222.0.42 () {60 vars in 1134 bytes} [Mon Sep 19 10:17:29 2022] POST /identity/v3/groups => generated 229 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:17:29.772568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f1b32-2ebe-4606-8b55-76862ae41078 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:29.818479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c4025-214c-4823-9b25-d926cc1fe7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:29.819207 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c4025-214c-4823-9b25-d926cc1fe7db None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:29.819547 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c4025-214c-4823-9b25-d926cc1fe7db None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:29.819887 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c4025-214c-4823-9b25-d926cc1fe7db None admin] PATH_INFO: `/v3/users/05a7ad8095e84d11ad8d9a2cd8e9bed3` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:29.837530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c4025-214c-4823-9b25-d926cc1fe7db None admin] RBAC: Authorizing `identity:get_user(user_id=05a7ad8095e84d11ad8d9a2cd8e9bed3)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:29.839086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c4025-214c-4823-9b25-d926cc1fe7db None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:29.845515 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 153/306] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:29 2022] GET /identity/v3/users/05a7ad8095e84d11ad8d9a2cd8e9bed3 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:29.847432 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c4025-214c-4823-9b25-d926cc1fe7db None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:29.921484 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f1b32-2ebe-4606-8b55-76862ae41078 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:29.952705 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:29.966633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4dd560-b931-4e1f-bfc6-6bfdd29a15c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:29.967898 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dd560-b931-4e1f-bfc6-6bfdd29a15c0 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:29.968183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dd560-b931-4e1f-bfc6-6bfdd29a15c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:29.968485 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dd560-b931-4e1f-bfc6-6bfdd29a15c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:29.969741 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dd560-b931-4e1f-bfc6-6bfdd29a15c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:29.970060 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dd560-b931-4e1f-bfc6-6bfdd29a15c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:29.972052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dd560-b931-4e1f-bfc6-6bfdd29a15c0 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:29.972824 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c4025-214c-4823-9b25-d926cc1fe7db None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:29.992088 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 154/307] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:29.996435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4dd560-b931-4e1f-bfc6-6bfdd29a15c0 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:30.004917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:30.019748 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f6fc0-8403-48fa-8096-3fbec82e2fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:30.020353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f6fc0-8403-48fa-8096-3fbec82e2fbc None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:30.020624 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f6fc0-8403-48fa-8096-3fbec82e2fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:30.020889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f6fc0-8403-48fa-8096-3fbec82e2fbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:30.021650 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f6fc0-8403-48fa-8096-3fbec82e2fbc None admin] RBAC: Adding query filter params (role.id=c1c0bce9f7a34def89550deb2e26113a, user.id=05a7ad8095e84d11ad8d9a2cd8e9bed3, scope.system=all) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:30.021962 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f6fc0-8403-48fa-8096-3fbec82e2fbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:30.023377 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f6fc0-8403-48fa-8096-3fbec82e2fbc None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:30.029390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 154/308] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Sep 19 10:17:29 2022] GET /identity/v3/role_assignments?user.id=05a7ad8095e84d11ad8d9a2cd8e9bed3&scope.system=all&role.id=c1c0bce9f7a34def89550deb2e26113a => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:30.125154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4dd560-b931-4e1f-bfc6-6bfdd29a15c0 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:30.164642 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c53fba-8a40-4272-817e-eca7072ddc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:30.165332 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c53fba-8a40-4272-817e-eca7072ddc9a None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:30.165651 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c53fba-8a40-4272-817e-eca7072ddc9a None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:30.165937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c53fba-8a40-4272-817e-eca7072ddc9a None admin] PATH_INFO: `/v3/users/e7f9dab65285481b9dd18023a350a6cf` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:30.167116 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c53fba-8a40-4272-817e-eca7072ddc9a None admin] RBAC: Authorizing `identity:get_user(user_id=e7f9dab65285481b9dd18023a350a6cf)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:30.168586 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c53fba-8a40-4272-817e-eca7072ddc9a None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:30.170654 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 155/309] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:29 2022] GET /identity/v3/users/e7f9dab65285481b9dd18023a350a6cf => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:30.176103 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f6fc0-8403-48fa-8096-3fbec82e2fbc None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:30.303605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f6fc0-8403-48fa-8096-3fbec82e2fbc None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:30.323768 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884dc37-21c3-4a8b-8ce5-12ec1cb79a6e None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:30.324084 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884dc37-21c3-4a8b-8ce5-12ec1cb79a6e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:30.324311 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884dc37-21c3-4a8b-8ce5-12ec1cb79a6e None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:30.325803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 156/310] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:17:30.331857 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd22712-5107-401f-afe6-73c2e467fa40 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:30.332139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd22712-5107-401f-afe6-73c2e467fa40 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:30.332415 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd22712-5107-401f-afe6-73c2e467fa40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:30.346796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a9c77-568c-4ecf-94a7-c9615fb2b056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:30.347452 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a9c77-568c-4ecf-94a7-c9615fb2b056 None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:30.347727 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a9c77-568c-4ecf-94a7-c9615fb2b056 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:30.348001 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a9c77-568c-4ecf-94a7-c9615fb2b056 None admin] PATH_INFO: `/v3/system/users/e7f9dab65285481b9dd18023a350a6cf/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:30.349772 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a9c77-568c-4ecf-94a7-c9615fb2b056 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e7f9dab65285481b9dd18023a350a6cf, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:30.351210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a9c77-568c-4ecf-94a7-c9615fb2b056 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:30.353326 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-9fd22712-5107-401f-afe6-73c2e467fa40 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:30.354902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-9fd22712-5107-401f-afe6-73c2e467fa40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:30.411314 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:30.428729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 157/311] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:17:30.435019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 155/312] 10.222.0.42 () {58 vars in 1334 bytes} [Mon Sep 19 10:17:30 2022] PUT /identity/v3/system/users/e7f9dab65285481b9dd18023a350a6cf/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:17:30.717668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd22712-5107-401f-afe6-73c2e467fa40 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:30.828380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd22712-5107-401f-afe6-73c2e467fa40 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:30.864094 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71af986-8683-4d2c-bbab-914ea5f31a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:30.864738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71af986-8683-4d2c-bbab-914ea5f31a8d None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:30.865017 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71af986-8683-4d2c-bbab-914ea5f31a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:30.865316 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71af986-8683-4d2c-bbab-914ea5f31a8d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:30.876321 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71af986-8683-4d2c-bbab-914ea5f31a8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:30.881348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-e71af986-8683-4d2c-bbab-914ea5f31a8d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 10:17:30.881348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:30.881348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:30.881348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:30.881348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:30.881348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:30.881348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:30.881348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:30.881348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:30.881348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:30.881348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:30.881348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:30.881348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:30.881348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:30.881348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:30.881348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:30.882249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:30.882249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:30.882249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:30.882249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:30.882249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:30.882249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:30.882249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:30.882249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:30.882249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:30.882249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:30.882249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:30.882249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:30.882249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:30.882249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:30.882249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:30.882249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:30.883461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:30.883461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:30.883461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:30.883461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:30.883461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:30.883461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:30.883461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:30.883461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:30.883461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:30.883461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:30.883461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 10:17:30.883461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:30.883461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 158/313] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:30.888482 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a9c77-568c-4ecf-94a7-c9615fb2b056 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:30.948281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:30.994039 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a9c77-568c-4ecf-94a7-c9615fb2b056 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:31.028358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f8fe0-14bf-4586-9453-2e32daa19974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:31.028974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f8fe0-14bf-4586-9453-2e32daa19974 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:31.029280 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f8fe0-14bf-4586-9453-2e32daa19974 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:31.029630 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f8fe0-14bf-4586-9453-2e32daa19974 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:31.030293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f8fe0-14bf-4586-9453-2e32daa19974 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:31.030639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f8fe0-14bf-4586-9453-2e32daa19974 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:31.032067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f8fe0-14bf-4586-9453-2e32daa19974 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:31.040340 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 156/314] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:31.046381 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71af986-8683-4d2c-bbab-914ea5f31a8d None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:31.103336 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:31.159991 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71af986-8683-4d2c-bbab-914ea5f31a8d None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:31.197474 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6b799-c827-488e-921a-638f6c5cdf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:31.198105 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6b799-c827-488e-921a-638f6c5cdf69 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:31.198378 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6b799-c827-488e-921a-638f6c5cdf69 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:31.198703 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6b799-c827-488e-921a-638f6c5cdf69 None admin] PATH_INFO: `/v3/users/a4884d1729914f048141ea8e9576ea49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:31.199823 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6b799-c827-488e-921a-638f6c5cdf69 None admin] RBAC: Authorizing `identity:get_user(user_id=a4884d1729914f048141ea8e9576ea49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:31.201841 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6b799-c827-488e-921a-638f6c5cdf69 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:31.204900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 159/315] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:31 2022] GET /identity/v3/users/a4884d1729914f048141ea8e9576ea49 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:31.211467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f8fe0-14bf-4586-9453-2e32daa19974 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:31.317200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f8fe0-14bf-4586-9453-2e32daa19974 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:31.335754 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:31.353792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71961a3-b96a-48f6-8d09-da4cf9b53bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:31.356405 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71961a3-b96a-48f6-8d09-da4cf9b53bad None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:31.356405 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71961a3-b96a-48f6-8d09-da4cf9b53bad None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:31.356405 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71961a3-b96a-48f6-8d09-da4cf9b53bad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:31.423526 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71961a3-b96a-48f6-8d09-da4cf9b53bad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:31.425145 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71961a3-b96a-48f6-8d09-da4cf9b53bad None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:31.525583 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 157/316] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:31.531119 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6b799-c827-488e-921a-638f6c5cdf69 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:31.632838 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6b799-c827-488e-921a-638f6c5cdf69 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:31.646257 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:31.667970 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acd7fb-c550-4f85-8ab0-b7c9de20e1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:31.676942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acd7fb-c550-4f85-8ab0-b7c9de20e1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:31.676942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acd7fb-c550-4f85-8ab0-b7c9de20e1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:31.676942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acd7fb-c550-4f85-8ab0-b7c9de20e1a3 None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:31.676942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acd7fb-c550-4f85-8ab0-b7c9de20e1a3 None admin] RBAC: Authorizing `identity:get_project(project_id=a52b1cb46c36451498d5787d735e4f12)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:31.676942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acd7fb-c550-4f85-8ab0-b7c9de20e1a3 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:31.676942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 160/317] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:31 2022] GET /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:31.680172 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71961a3-b96a-48f6-8d09-da4cf9b53bad None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:31.709465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:31.784915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71961a3-b96a-48f6-8d09-da4cf9b53bad None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:31.819362 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf871f-a3ab-42bc-a949-ba2316604f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:31.819977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf871f-a3ab-42bc-a949-ba2316604f92 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:31.820252 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf871f-a3ab-42bc-a949-ba2316604f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:31.820523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf871f-a3ab-42bc-a949-ba2316604f92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:31.821287 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf871f-a3ab-42bc-a949-ba2316604f92 None admin] RBAC: Adding query filter params (role.id=c1c0bce9f7a34def89550deb2e26113a, scope.project.id=a52b1cb46c36451498d5787d735e4f12, user.id=a4884d1729914f048141ea8e9576ea49) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:31.821601 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf871f-a3ab-42bc-a949-ba2316604f92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:31.823028 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf871f-a3ab-42bc-a949-ba2316604f92 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:31.834983 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 158/318] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:17:31 2022] GET /identity/v3/role_assignments?user.id=a4884d1729914f048141ea8e9576ea49&scope.project.id=a52b1cb46c36451498d5787d735e4f12&role.id=c1c0bce9f7a34def89550deb2e26113a => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:32.161962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8cc3a-f1e3-4677-801b-10223c676eec None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:32.162504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8cc3a-f1e3-4677-801b-10223c676eec None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:32.162504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8cc3a-f1e3-4677-801b-10223c676eec None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:32.163845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 161/319] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:32.169358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d10d0-42ab-40b6-b89a-9578b2e68063 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:32.169816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d10d0-42ab-40b6-b89a-9578b2e68063 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:32.170270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d10d0-42ab-40b6-b89a-9578b2e68063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:32.188759 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-728d10d0-42ab-40b6-b89a-9578b2e68063 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:32.190212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-728d10d0-42ab-40b6-b89a-9578b2e68063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:32.196418 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce664236-e07a-455c-8c09-1c0db36d0a8d None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:32.196705 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce664236-e07a-455c-8c09-1c0db36d0a8d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:32.196981 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce664236-e07a-455c-8c09-1c0db36d0a8d None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:32.198642 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 162/320] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:32.204029 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442ced5-9219-43ae-8cf6-1dd6ca295ce7 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:32.204313 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442ced5-9219-43ae-8cf6-1dd6ca295ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:32.204587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442ced5-9219-43ae-8cf6-1dd6ca295ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:32.222329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-5442ced5-9219-43ae-8cf6-1dd6ca295ce7 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:32.223285 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5442ced5-9219-43ae-8cf6-1dd6ca295ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:32.262430 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 159/321] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:32.292698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 163/322] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:17:32.540123 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d10d0-42ab-40b6-b89a-9578b2e68063 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:32.552082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:32.560291 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442ced5-9219-43ae-8cf6-1dd6ca295ce7 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:32.646347 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d10d0-42ab-40b6-b89a-9578b2e68063 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:32.661757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442ced5-9219-43ae-8cf6-1dd6ca295ce7 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:32.672204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:32.680369 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a8089-9687-4618-8d7a-bbe224cd7ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:32.687209 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a8089-9687-4618-8d7a-bbe224cd7ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:32.687209 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a8089-9687-4618-8d7a-bbe224cd7ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:32.687209 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a8089-9687-4618-8d7a-bbe224cd7ab5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:32.687209 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a8089-9687-4618-8d7a-bbe224cd7ab5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:32.687209 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a8089-9687-4618-8d7a-bbe224cd7ab5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:32.687209 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 160/323] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:17:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:32.687600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1373d-a9be-44ed-8cdf-8c95f5079b2b None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:32.687600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1373d-a9be-44ed-8cdf-8c95f5079b2b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:32.687600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1373d-a9be-44ed-8cdf-8c95f5079b2b None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:32.688823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 161/324] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:32.691553 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1373d-a9be-44ed-8cdf-8c95f5079b2b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:32.696039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc05bb-7b2b-4f0c-b090-9fa3257a1733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:32.696670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc05bb-7b2b-4f0c-b090-9fa3257a1733 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:32.696949 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc05bb-7b2b-4f0c-b090-9fa3257a1733 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:32.697251 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc05bb-7b2b-4f0c-b090-9fa3257a1733 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:32.707659 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc05bb-7b2b-4f0c-b090-9fa3257a1733 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:32.712936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-aabc05bb-7b2b-4f0c-b090-9fa3257a1733 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 10:17:32.712936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:32.712936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:32.712936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:32.712936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:32.712936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:32.712936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:32.712936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:32.712936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:32.712936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:32.712936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:32.712936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:32.712936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:32.712936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:32.712936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:32.712936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:32.713999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:32.713999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:32.713999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:32.713999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:32.713999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:32.713999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:32.713999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:32.713999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:32.713999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:32.713999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:32.713999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:32.713999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:32.713999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:32.713999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:32.713999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:32.713999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:32.714802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:32.714802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:32.714802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:32.714802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:32.714802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:32.714802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:32.714802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:32.714802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:32.714802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:32.714802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:32.714802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 10:17:32.714802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:32.714802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 164/325] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:17:32.718625 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39a466-2b9e-445a-a1e4-f50ff024d2f1 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:32.718907 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39a466-2b9e-445a-a1e4-f50ff024d2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:32.719182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39a466-2b9e-445a-a1e4-f50ff024d2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:32.736428 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-4f39a466-2b9e-445a-a1e4-f50ff024d2f1 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:32.737634 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4f39a466-2b9e-445a-a1e4-f50ff024d2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:32.774128 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:32.796117 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1373d-a9be-44ed-8cdf-8c95f5079b2b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:32.804600 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 165/326] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:32.805472 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39a466-2b9e-445a-a1e4-f50ff024d2f1 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:32.833095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b985e6-6efc-4c16-89f2-a9998bc42296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:32.833713 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b985e6-6efc-4c16-89f2-a9998bc42296 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:32.833990 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b985e6-6efc-4c16-89f2-a9998bc42296 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:32.834265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b985e6-6efc-4c16-89f2-a9998bc42296 None admin] PATH_INFO: `/v3/groups` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:32.834893 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b985e6-6efc-4c16-89f2-a9998bc42296 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:32.836579 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b985e6-6efc-4c16-89f2-a9998bc42296 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:32.915379 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:32.918547 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39a466-2b9e-445a-a1e4-f50ff024d2f1 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:32.933537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 162/327] 10.222.0.42 () {60 vars in 1134 bytes} [Mon Sep 19 10:17:32 2022] POST /identity/v3/groups => generated 226 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:17:32.955311 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250bdcd-6936-4b83-8f7a-4fc3db0ade72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:32.955936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250bdcd-6936-4b83-8f7a-4fc3db0ade72 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:32.956606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250bdcd-6936-4b83-8f7a-4fc3db0ade72 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:32.957257 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250bdcd-6936-4b83-8f7a-4fc3db0ade72 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:32.958160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250bdcd-6936-4b83-8f7a-4fc3db0ade72 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:32.958942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250bdcd-6936-4b83-8f7a-4fc3db0ade72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:32.960385 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250bdcd-6936-4b83-8f7a-4fc3db0ade72 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:32.974581 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 166/328] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:32.975859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b985e6-6efc-4c16-89f2-a9998bc42296 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:33.073891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250bdcd-6936-4b83-8f7a-4fc3db0ade72 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:33.087143 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b985e6-6efc-4c16-89f2-a9998bc42296 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:33.138972 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a46f6ac-a72b-40ac-b75f-52bdb4300c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:33.139866 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a46f6ac-a72b-40ac-b75f-52bdb4300c3b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:33.140244 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a46f6ac-a72b-40ac-b75f-52bdb4300c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:33.140526 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a46f6ac-a72b-40ac-b75f-52bdb4300c3b None admin] PATH_INFO: `/v3/users/05a7ad8095e84d11ad8d9a2cd8e9bed3` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:33.141832 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a46f6ac-a72b-40ac-b75f-52bdb4300c3b None admin] RBAC: Authorizing `identity:get_user(user_id=05a7ad8095e84d11ad8d9a2cd8e9bed3)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:33.143389 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a46f6ac-a72b-40ac-b75f-52bdb4300c3b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:33.145029 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 163/329] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:32 2022] GET /identity/v3/users/05a7ad8095e84d11ad8d9a2cd8e9bed3 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:33.151293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a46f6ac-a72b-40ac-b75f-52bdb4300c3b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:33.164442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:33.195505 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250bdcd-6936-4b83-8f7a-4fc3db0ade72 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:33.236859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92ff2c-06a1-4c23-be5b-b5c700426bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:33.238729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92ff2c-06a1-4c23-be5b-b5c700426bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:33.239303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92ff2c-06a1-4c23-be5b-b5c700426bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:33.240712 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92ff2c-06a1-4c23-be5b-b5c700426bb0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:33.254563 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92ff2c-06a1-4c23-be5b-b5c700426bb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:33.260765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-5c92ff2c-06a1-4c23-be5b-b5c700426bb0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:33.260765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:33.260765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:33.260765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:33.260765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:33.260765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:33.260765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:33.260765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:33.260765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:33.260765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:33.260765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:33.260765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:33.260765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:33.260765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:33.260765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:33.260765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:33.261688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:33.261688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:33.261688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:33.261688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:33.261688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:33.261688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:33.261688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:33.261688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:33.261688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:33.261688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:33.261688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:33.261688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:33.261688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:33.261688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:33.261688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:33.261688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:33.262513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:33.262513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:33.262513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:33.262513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:33.262513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:33.262513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:33.262513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:33.262513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:33.262513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:33.262513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:33.262513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:33.262513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:33.262513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 167/330] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:33.270122 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92ff2c-06a1-4c23-be5b-b5c700426bb0 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:33.292479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a46f6ac-a72b-40ac-b75f-52bdb4300c3b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:33.300474 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:33.349649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7045e3-cc65-4e90-8f2e-f3c691b8703f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:33.350230 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7045e3-cc65-4e90-8f2e-f3c691b8703f None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:33.350506 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7045e3-cc65-4e90-8f2e-f3c691b8703f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:33.350779 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7045e3-cc65-4e90-8f2e-f3c691b8703f None admin] PATH_INFO: `/v3/system/users/05a7ad8095e84d11ad8d9a2cd8e9bed3/roles/c1c0bce9f7a34def89550deb2e26113a` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:33.352319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7045e3-cc65-4e90-8f2e-f3c691b8703f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=05a7ad8095e84d11ad8d9a2cd8e9bed3, role_id=c1c0bce9f7a34def89550deb2e26113a)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:33.353739 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7045e3-cc65-4e90-8f2e-f3c691b8703f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:33.390732 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92ff2c-06a1-4c23-be5b-b5c700426bb0 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:33.409084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 164/331] 10.222.0.42 () {58 vars in 1334 bytes} [Mon Sep 19 10:17:33 2022] PUT /identity/v3/system/users/05a7ad8095e84d11ad8d9a2cd8e9bed3/roles/c1c0bce9f7a34def89550deb2e26113a => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:17:33.437231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d9bfb-7c83-46bf-aaba-479bbd57227c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:33.437971 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d9bfb-7c83-46bf-aaba-479bbd57227c None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:33.438302 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d9bfb-7c83-46bf-aaba-479bbd57227c None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:33.438575 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d9bfb-7c83-46bf-aaba-479bbd57227c None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:33.439328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d9bfb-7c83-46bf-aaba-479bbd57227c None admin] RBAC: Adding query filter params (name=member) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:33.439694 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d9bfb-7c83-46bf-aaba-479bbd57227c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:33.441225 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d9bfb-7c83-46bf-aaba-479bbd57227c None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:33.452294 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 168/332] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:33.459777 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7045e3-cc65-4e90-8f2e-f3c691b8703f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:33.552452 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:33.587681 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7045e3-cc65-4e90-8f2e-f3c691b8703f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:33.624699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b11aef-91c4-49b2-b65a-01e78d7f9f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:33.625288 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b11aef-91c4-49b2-b65a-01e78d7f9f69 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:33.625563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b11aef-91c4-49b2-b65a-01e78d7f9f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:33.625839 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b11aef-91c4-49b2-b65a-01e78d7f9f69 None admin] PATH_INFO: `/v3/users/e7f9dab65285481b9dd18023a350a6cf` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:33.627005 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b11aef-91c4-49b2-b65a-01e78d7f9f69 None admin] RBAC: Authorizing `identity:get_user(user_id=e7f9dab65285481b9dd18023a350a6cf)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:33.628466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b11aef-91c4-49b2-b65a-01e78d7f9f69 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:33.630393 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 165/333] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:33 2022] GET /identity/v3/users/e7f9dab65285481b9dd18023a350a6cf => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:33.636268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d9bfb-7c83-46bf-aaba-479bbd57227c None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:33.740295 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:33.759228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d9bfb-7c83-46bf-aaba-479bbd57227c None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:33.797470 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7a724-b50d-4e59-92a7-65b3e2f065a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:33.798085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7a724-b50d-4e59-92a7-65b3e2f065a8 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:33.798365 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7a724-b50d-4e59-92a7-65b3e2f065a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:33.798641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7a724-b50d-4e59-92a7-65b3e2f065a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:33.799385 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7a724-b50d-4e59-92a7-65b3e2f065a8 None admin] RBAC: Adding query filter params (role.id=c44d082d88894b888e94797acf7655b7, user.id=e7f9dab65285481b9dd18023a350a6cf, scope.system=all) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:33.799688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7a724-b50d-4e59-92a7-65b3e2f065a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:33.801164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7a724-b50d-4e59-92a7-65b3e2f065a8 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:33.814754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 169/334] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Sep 19 10:17:33 2022] GET /identity/v3/role_assignments?user.id=e7f9dab65285481b9dd18023a350a6cf&scope.system=all&role.id=c44d082d88894b888e94797acf7655b7 => generated 510 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:35.422516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b58a93-fcd6-4ce4-92c9-7429754ac701 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:35.423050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b58a93-fcd6-4ce4-92c9-7429754ac701 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:35.423050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b58a93-fcd6-4ce4-92c9-7429754ac701 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:35.424897 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 166/335] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:35.430650 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b8fba-004d-434b-b819-60b77b895443 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:35.434074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b8fba-004d-434b-b819-60b77b895443 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:35.434074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b8fba-004d-434b-b819-60b77b895443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:35.452619 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-b22b8fba-004d-434b-b819-60b77b895443 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:35.453854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b22b8fba-004d-434b-b819-60b77b895443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:35.541685 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 170/336] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:35.890715 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b58a93-fcd6-4ce4-92c9-7429754ac701 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:35.982457 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49885dd-86a5-4f68-86e1-feb11f8235ac None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:35.982791 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49885dd-86a5-4f68-86e1-feb11f8235ac None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:35.983012 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49885dd-86a5-4f68-86e1-feb11f8235ac None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:35.984426 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 171/337] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:35.989628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d8810-b85f-46e8-a0a2-0634bda045f9 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:35.990144 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d8810-b85f-46e8-a0a2-0634bda045f9 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:35.990617 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d8810-b85f-46e8-a0a2-0634bda045f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:36.029586 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b58a93-fcd6-4ce4-92c9-7429754ac701 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:36.030888 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-bd5d8810-b85f-46e8-a0a2-0634bda045f9 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:36.032292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-bd5d8810-b85f-46e8-a0a2-0634bda045f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:36.160262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994f210-b4fa-467c-b54e-e4b7d6d1ab2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:36.160915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994f210-b4fa-467c-b54e-e4b7d6d1ab2f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:36.161240 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994f210-b4fa-467c-b54e-e4b7d6d1ab2f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:36.161516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994f210-b4fa-467c-b54e-e4b7d6d1ab2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:36.176413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:36.193545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994f210-b4fa-467c-b54e-e4b7d6d1ab2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:36.204328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-5994f210-b4fa-467c-b54e-e4b7d6d1ab2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:36.204328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:36.204328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:36.204328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:36.204328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:36.204328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:36.204328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:36.204328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:36.204328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:36.204328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:36.204328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:36.204328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:36.204328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:36.204328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:36.204328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:36.204328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:36.205286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:36.205286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:36.205286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:36.205286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:36.205286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:36.205286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:36.205286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:36.205286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:36.205286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:36.205286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:36.205286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:36.205286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:36.205286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:36.205286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:36.205286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:36.205286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:36.206086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:36.206086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:36.206086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:36.206086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:36.206086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:36.206086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:36.206086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:36.206086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:36.206086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:36.206086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:36.206086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:36.206086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:36.208725 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 167/338] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:36.216749 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994f210-b4fa-467c-b54e-e4b7d6d1ab2f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:36.295363 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 172/339] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:36.336688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f26140-dedf-40fa-a37d-05fc41b36aa9 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:36.336688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f26140-dedf-40fa-a37d-05fc41b36aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:36.336688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f26140-dedf-40fa-a37d-05fc41b36aa9 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:36.336688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 173/340] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:36.342904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2e292-37e4-44df-b79c-b1dfa95d467b None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:36.343456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2e292-37e4-44df-b79c-b1dfa95d467b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:36.343929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2e292-37e4-44df-b79c-b1dfa95d467b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:36.365467 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:36.389078 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-d2a2e292-37e4-44df-b79c-b1dfa95d467b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:36.390649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d2a2e292-37e4-44df-b79c-b1dfa95d467b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:36.539481 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 174/341] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:36.575029 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994f210-b4fa-467c-b54e-e4b7d6d1ab2f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:36.658698 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afe4d6-56bf-4dea-8fa5-61eace0f5de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:36.659322 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afe4d6-56bf-4dea-8fa5-61eace0f5de1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:36.659606 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afe4d6-56bf-4dea-8fa5-61eace0f5de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:36.659887 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afe4d6-56bf-4dea-8fa5-61eace0f5de1 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:36.660554 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afe4d6-56bf-4dea-8fa5-61eace0f5de1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:36.660860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afe4d6-56bf-4dea-8fa5-61eace0f5de1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:36.662262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afe4d6-56bf-4dea-8fa5-61eace0f5de1 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:36.680619 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 168/342] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 464 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:36.687114 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a2e292-37e4-44df-b79c-b1dfa95d467b None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:36.738149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afe4d6-56bf-4dea-8fa5-61eace0f5de1 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:36.917016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:36.935483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a2e292-37e4-44df-b79c-b1dfa95d467b None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:36.985441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:36.990074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57a395-9b9b-466b-9ae1-7a31ce815b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:36.990688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57a395-9b9b-466b-9ae1-7a31ce815b6d None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:36.990966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57a395-9b9b-466b-9ae1-7a31ce815b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:36.991234 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57a395-9b9b-466b-9ae1-7a31ce815b6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:36.991348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afe4d6-56bf-4dea-8fa5-61eace0f5de1 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:37.060923 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfae396-bccf-4d73-b251-2bc2a59e9837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:37.061599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfae396-bccf-4d73-b251-2bc2a59e9837 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:37.061878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfae396-bccf-4d73-b251-2bc2a59e9837 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:37.062152 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfae396-bccf-4d73-b251-2bc2a59e9837 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:37.083933 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfae396-bccf-4d73-b251-2bc2a59e9837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:37.090008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-7bfae396-bccf-4d73-b251-2bc2a59e9837 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 10:17:37.090008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:37.090008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:37.090008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:37.090008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:37.090008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:37.090008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:37.090008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:37.090008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:37.090008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:37.090008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:37.090008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:37.090008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:37.090008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:37.090008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:37.090008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:37.090912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:37.090912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:37.090912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:37.090912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:37.090912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:37.090912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:37.090912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:37.090912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:37.090912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:37.090912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:37.090912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:37.090912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:37.090912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:37.090912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:37.090912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:37.090912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:37.091746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:37.091746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:37.091746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:37.091746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:37.091746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:37.091746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:37.091746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:37.091746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:37.091746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:37.091746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:37.091746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 10:17:37.091746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:37.097128 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 169/343] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:37.098242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfae396-bccf-4d73-b251-2bc2a59e9837 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:37.102209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57a395-9b9b-466b-9ae1-7a31ce815b6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:37.104723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57a395-9b9b-466b-9ae1-7a31ce815b6d None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:37.237126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 175/344] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:37.238238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57a395-9b9b-466b-9ae1-7a31ce815b6d None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:37.298681 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:37.320712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfae396-bccf-4d73-b251-2bc2a59e9837 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:37.356118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:37.385033 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5531d-d5f1-43d9-92ff-003b966f0956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:37.386453 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5531d-d5f1-43d9-92ff-003b966f0956 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:37.387052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5531d-d5f1-43d9-92ff-003b966f0956 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:37.387486 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5531d-d5f1-43d9-92ff-003b966f0956 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:37.407063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5531d-d5f1-43d9-92ff-003b966f0956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:37.413249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57a395-9b9b-466b-9ae1-7a31ce815b6d None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:37.413811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-11a5531d-d5f1-43d9-92ff-003b966f0956 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:37.413811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:37.413811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:37.413811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:37.413811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:37.413811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:37.413811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:37.413811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:37.413811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:37.413811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:37.413811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:37.413811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:37.413811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:37.413811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:37.413811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:37.413811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:37.414985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:37.414985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:37.414985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:37.414985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:37.414985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:37.414985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:37.414985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:37.414985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:37.414985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:37.414985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:37.414985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:37.414985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:37.414985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:37.414985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:37.414985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:37.414985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:37.416008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:37.416008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:37.416008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:37.416008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:37.416008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:37.416008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:37.416008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:37.416008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:37.416008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:37.416008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:37.416008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:37.416008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:37.416008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 170/345] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:17:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:37.416008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5531d-d5f1-43d9-92ff-003b966f0956 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:37.482006 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4973fd-6695-41e5-af3f-e9164dc9ccad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:37.483478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4973fd-6695-41e5-af3f-e9164dc9ccad None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:37.483758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4973fd-6695-41e5-af3f-e9164dc9ccad None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:37.484033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4973fd-6695-41e5-af3f-e9164dc9ccad None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:37.484855 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4973fd-6695-41e5-af3f-e9164dc9ccad None admin] RBAC: Adding query filter params (name=reader) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:37.485209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4973fd-6695-41e5-af3f-e9164dc9ccad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:37.487634 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4973fd-6695-41e5-af3f-e9164dc9ccad None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:37.503867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 176/346] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:37.505443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5802b-bcf1-452e-b7f5-18eaf5fda63d None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:37.505877 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5802b-bcf1-452e-b7f5-18eaf5fda63d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:37.506300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5802b-bcf1-452e-b7f5-18eaf5fda63d None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:37.507691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 177/347] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:37.512901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5802b-bcf1-452e-b7f5-18eaf5fda63d None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:37.552141 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5531d-d5f1-43d9-92ff-003b966f0956 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:37.597446 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530974e-501e-435d-b063-c4f695880bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:37.598757 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530974e-501e-435d-b063-c4f695880bca None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:37.599039 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530974e-501e-435d-b063-c4f695880bca None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:37.599496 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530974e-501e-435d-b063-c4f695880bca None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:37.600640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530974e-501e-435d-b063-c4f695880bca None admin] RBAC: Authorizing `identity:get_project(project_id=8959cf36bec74b8ca22049270ee47cb2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:37.602949 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530974e-501e-435d-b063-c4f695880bca None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:37.604751 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 171/348] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:37 2022] GET /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:37.605452 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530974e-501e-435d-b063-c4f695880bca None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:37.637772 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:37.646936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5802b-bcf1-452e-b7f5-18eaf5fda63d None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:37.663664 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:37.694746 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487faaf3-771f-45b1-9f6c-d412eea5b3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:37.695376 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487faaf3-771f-45b1-9f6c-d412eea5b3ca None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:37.695658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487faaf3-771f-45b1-9f6c-d412eea5b3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:37.695934 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487faaf3-771f-45b1-9f6c-d412eea5b3ca None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:37.696607 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487faaf3-771f-45b1-9f6c-d412eea5b3ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:37.696908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487faaf3-771f-45b1-9f6c-d412eea5b3ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:37.698359 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487faaf3-771f-45b1-9f6c-d412eea5b3ca None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:37.712179 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 178/349] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:17:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:37.713463 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffe1a7-1756-4a00-ab1d-96fed2aa7feb None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:37.713748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffe1a7-1756-4a00-ab1d-96fed2aa7feb None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:37.714024 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffe1a7-1756-4a00-ab1d-96fed2aa7feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:37.725812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530974e-501e-435d-b063-c4f695880bca None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:37.739557 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-39ffe1a7-1756-4a00-ab1d-96fed2aa7feb None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:37.740770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-39ffe1a7-1756-4a00-ab1d-96fed2aa7feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:37.773147 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a5fca-cd86-4947-836c-8c83a2d40840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:37.774233 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a5fca-cd86-4947-836c-8c83a2d40840 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:37.774972 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a5fca-cd86-4947-836c-8c83a2d40840 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:37.775431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a5fca-cd86-4947-836c-8c83a2d40840 None admin] PATH_INFO: `/v3/users/05a7ad8095e84d11ad8d9a2cd8e9bed3` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:37.776616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a5fca-cd86-4947-836c-8c83a2d40840 None admin] RBAC: Authorizing `identity:get_user(user_id=05a7ad8095e84d11ad8d9a2cd8e9bed3)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:37.778080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a5fca-cd86-4947-836c-8c83a2d40840 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:37.779872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 172/350] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:17:37 2022] GET /identity/v3/users/05a7ad8095e84d11ad8d9a2cd8e9bed3 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:37.781094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130aa44-31c9-4999-be3e-024c656b89d6 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:37.781094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130aa44-31c9-4999-be3e-024c656b89d6 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:37.781415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130aa44-31c9-4999-be3e-024c656b89d6 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:37.783051 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 173/351] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:37.783764 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130aa44-31c9-4999-be3e-024c656b89d6 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:37.825617 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 179/352] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:37.827275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffe1a7-1756-4a00-ab1d-96fed2aa7feb None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:37.930848 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:37.943921 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130aa44-31c9-4999-be3e-024c656b89d6 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:37.948685 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffe1a7-1756-4a00-ab1d-96fed2aa7feb None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:37.964070 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:37.995395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85426216-23d3-4fd0-9388-357c9b0e64f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:37.996050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85426216-23d3-4fd0-9388-357c9b0e64f6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:37.996330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85426216-23d3-4fd0-9388-357c9b0e64f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:37.996608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85426216-23d3-4fd0-9388-357c9b0e64f6 None admin] PATH_INFO: `/v3/groups/4d2c82261800420081b2b0721af10b12` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:38.007271 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85426216-23d3-4fd0-9388-357c9b0e64f6 None admin] RBAC: Authorizing `identity:get_group(group_id=4d2c82261800420081b2b0721af10b12)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:38.008279 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85426216-23d3-4fd0-9388-357c9b0e64f6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:38.009629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144f5aa-0277-41eb-956d-854f084de156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:38.010430 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 174/353] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 19 10:17:37 2022] GET /identity/v3/groups/4d2c82261800420081b2b0721af10b12 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:38.010701 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144f5aa-0277-41eb-956d-854f084de156 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:38.011171 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144f5aa-0277-41eb-956d-854f084de156 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:38.011288 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85426216-23d3-4fd0-9388-357c9b0e64f6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:38.018525 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144f5aa-0277-41eb-956d-854f084de156 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:38.100969 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144f5aa-0277-41eb-956d-854f084de156 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:38.102847 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144f5aa-0277-41eb-956d-854f084de156 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:38.130624 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85426216-23d3-4fd0-9388-357c9b0e64f6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:38.172060 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7a1fb-776b-4cf3-98fa-8dd1a5f8169e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:38.172672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7a1fb-776b-4cf3-98fa-8dd1a5f8169e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:38.172956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7a1fb-776b-4cf3-98fa-8dd1a5f8169e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:38.173264 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7a1fb-776b-4cf3-98fa-8dd1a5f8169e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:38.173994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7a1fb-776b-4cf3-98fa-8dd1a5f8169e None admin] RBAC: Adding query filter params (role.id=c1c0bce9f7a34def89550deb2e26113a, user.id=05a7ad8095e84d11ad8d9a2cd8e9bed3, scope.system=all) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:38.174303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7a1fb-776b-4cf3-98fa-8dd1a5f8169e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:38.175756 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7a1fb-776b-4cf3-98fa-8dd1a5f8169e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:38.188247 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 175/354] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Sep 19 10:17:38 2022] GET /identity/v3/role_assignments?user.id=05a7ad8095e84d11ad8d9a2cd8e9bed3&scope.system=all&role.id=c1c0bce9f7a34def89550deb2e26113a => generated 510 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:38.189442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715bb25-95af-4754-a210-6a6e6d1beebe None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:38.189740 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715bb25-95af-4754-a210-6a6e6d1beebe None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:38.190049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715bb25-95af-4754-a210-6a6e6d1beebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:38.200700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:38.210362 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6715bb25-95af-4754-a210-6a6e6d1beebe None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:38.211768 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6715bb25-95af-4754-a210-6a6e6d1beebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:38.214830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 180/355] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:38.215720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144f5aa-0277-41eb-956d-854f084de156 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:38.257607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:38.306962 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 176/356] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:38.308279 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a73f5-6668-4458-af14-080453b6435f None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:38.308565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a73f5-6668-4458-af14-080453b6435f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:38.308843 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a73f5-6668-4458-af14-080453b6435f None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:38.313287 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 177/357] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:38.313287 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a73f5-6668-4458-af14-080453b6435f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:38.378618 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144f5aa-0277-41eb-956d-854f084de156 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:38.440739 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4b733-4a16-4215-9487-9dc975a9b676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:38.441808 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4b733-4a16-4215-9487-9dc975a9b676 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:38.442277 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4b733-4a16-4215-9487-9dc975a9b676 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:38.442720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4b733-4a16-4215-9487-9dc975a9b676 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:38.443976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4b733-4a16-4215-9487-9dc975a9b676 None admin] RBAC: Adding query filter params (group.id=4d2c82261800420081b2b0721af10b12, role.id=c44d082d88894b888e94797acf7655b7, scope.project.id=8959cf36bec74b8ca22049270ee47cb2) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:38.444486 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4b733-4a16-4215-9487-9dc975a9b676 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:38.446962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4b733-4a16-4215-9487-9dc975a9b676 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:38.456308 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 181/358] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 19 10:17:38 2022] GET /identity/v3/role_assignments?group.id=4d2c82261800420081b2b0721af10b12&scope.project.id=8959cf36bec74b8ca22049270ee47cb2&role.id=c44d082d88894b888e94797acf7655b7 => generated 260 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:38.456730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4b733-4a16-4215-9487-9dc975a9b676 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:38.495800 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a73f5-6668-4458-af14-080453b6435f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:38.541051 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff1a8d-4601-4006-9740-2bacb3a94b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:38.542094 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:38.543521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff1a8d-4601-4006-9740-2bacb3a94b4d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:38.543963 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff1a8d-4601-4006-9740-2bacb3a94b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:38.544381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff1a8d-4601-4006-9740-2bacb3a94b4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:38.558168 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff1a8d-4601-4006-9740-2bacb3a94b4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:38.564597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-89ff1a8d-4601-4006-9740-2bacb3a94b4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:38.564597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:38.564597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:38.564597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:38.564597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:38.564597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:38.564597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:38.564597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:38.564597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:38.564597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:38.564597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:38.564597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:38.564597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:38.564597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:38.564597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:38.564597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:38.565812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:38.565812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:38.565812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:38.565812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:38.565812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:38.565812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:38.565812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:38.565812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:38.565812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:38.565812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:38.565812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:38.565812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:38.565812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:38.565812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:38.565812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:38.565812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:38.566802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:38.566802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:38.566802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:38.566802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:38.566802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:38.566802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:38.566802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:38.566802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:38.566802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:38.566802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:38.566802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:38.566802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:38.568911 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 178/359] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:38.570366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f95fdf-f075-41dc-b646-d2c589312411 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:38.570841 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f95fdf-f075-41dc-b646-d2c589312411 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:38.571258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f95fdf-f075-41dc-b646-d2c589312411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:38.596730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-90f95fdf-f075-41dc-b646-d2c589312411 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:38.599155 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-90f95fdf-f075-41dc-b646-d2c589312411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:38.610726 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4b733-4a16-4215-9487-9dc975a9b676 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:38.628634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:38.654546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7940121-7ab8-4bf8-9fad-3801b231821e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:38.655824 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7940121-7ab8-4bf8-9fad-3801b231821e None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:38.656110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7940121-7ab8-4bf8-9fad-3801b231821e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:38.656390 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7940121-7ab8-4bf8-9fad-3801b231821e None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:38.658169 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7940121-7ab8-4bf8-9fad-3801b231821e None admin] RBAC: Authorizing `identity:get_project(project_id=8959cf36bec74b8ca22049270ee47cb2)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:38.659648 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7940121-7ab8-4bf8-9fad-3801b231821e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:38.661445 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 182/360] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:38 2022] GET /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:38.662347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7940121-7ab8-4bf8-9fad-3801b231821e None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:38.691614 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 179/361] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:38.694626 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f95fdf-f075-41dc-b646-d2c589312411 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:38.795428 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7940121-7ab8-4bf8-9fad-3801b231821e None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:38.823199 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f95fdf-f075-41dc-b646-d2c589312411 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:38.826784 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:38.850663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea8d80-54d0-4b73-b993-59ecc314e23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:38.851621 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea8d80-54d0-4b73-b993-59ecc314e23e None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:38.851957 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea8d80-54d0-4b73-b993-59ecc314e23e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:38.852310 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea8d80-54d0-4b73-b993-59ecc314e23e None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:38.853436 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea8d80-54d0-4b73-b993-59ecc314e23e None admin] RBAC: Adding query filter params (name=member) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:38.854232 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea8d80-54d0-4b73-b993-59ecc314e23e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:38.856676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea8d80-54d0-4b73-b993-59ecc314e23e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:38.868399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 183/362] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:38.869226 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea8d80-54d0-4b73-b993-59ecc314e23e None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:38.875880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df283cd-f02f-463a-ac1e-7cdcdd781d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:38.876902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df283cd-f02f-463a-ac1e-7cdcdd781d4d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:38.877421 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df283cd-f02f-463a-ac1e-7cdcdd781d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:38.877889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df283cd-f02f-463a-ac1e-7cdcdd781d4d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:38.894706 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df283cd-f02f-463a-ac1e-7cdcdd781d4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:38.900463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-0df283cd-f02f-463a-ac1e-7cdcdd781d4d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:38.900463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:38.900463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:38.900463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:38.900463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:38.900463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:38.900463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:38.900463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:38.900463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:38.900463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:38.900463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:38.900463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:38.900463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:38.900463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:38.900463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:38.900463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:38.901390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:38.901390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:38.901390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:38.901390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:38.901390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:38.901390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:38.901390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:38.901390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:38.901390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:38.901390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:38.901390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:38.901390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:38.901390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:38.901390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:38.901390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:38.901390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:38.902193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:38.902193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:38.902193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:38.902193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:38.902193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:38.902193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:38.902193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:38.902193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:38.902193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:38.902193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:38.902193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:38.902193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:38.907003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 180/363] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:17:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:38.908053 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df283cd-f02f-463a-ac1e-7cdcdd781d4d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:38.933002 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:39.002610 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea8d80-54d0-4b73-b993-59ecc314e23e None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:39.029990 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df283cd-f02f-463a-ac1e-7cdcdd781d4d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:39.055894 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35568770-faac-43ad-88fd-185271817b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:39.056829 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35568770-faac-43ad-88fd-185271817b15 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:39.057215 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35568770-faac-43ad-88fd-185271817b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:39.057493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35568770-faac-43ad-88fd-185271817b15 None admin] PATH_INFO: `/v3/groups/4d2c82261800420081b2b0721af10b12` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:39.059286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35568770-faac-43ad-88fd-185271817b15 None admin] RBAC: Authorizing `identity:get_group(group_id=4d2c82261800420081b2b0721af10b12)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:39.061862 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35568770-faac-43ad-88fd-185271817b15 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:39.063372 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 184/364] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 19 10:17:38 2022] GET /identity/v3/groups/4d2c82261800420081b2b0721af10b12 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:39.068468 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35568770-faac-43ad-88fd-185271817b15 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:39.078241 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed58df-9ff6-4656-b92e-ce1a5c901c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:39.078854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed58df-9ff6-4656-b92e-ce1a5c901c10 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:39.079129 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed58df-9ff6-4656-b92e-ce1a5c901c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:39.079399 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed58df-9ff6-4656-b92e-ce1a5c901c10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:39.132989 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:39.172667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed58df-9ff6-4656-b92e-ce1a5c901c10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:39.174645 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed58df-9ff6-4656-b92e-ce1a5c901c10 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:39.197305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:39.198455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35568770-faac-43ad-88fd-185271817b15 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:39.243713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793eac9-8ac0-4830-a0a8-792173364303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:39.244719 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793eac9-8ac0-4830-a0a8-792173364303 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:39.245231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793eac9-8ac0-4830-a0a8-792173364303 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:39.245654 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793eac9-8ac0-4830-a0a8-792173364303 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:39.246639 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793eac9-8ac0-4830-a0a8-792173364303 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:39.247102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793eac9-8ac0-4830-a0a8-792173364303 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:39.249589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793eac9-8ac0-4830-a0a8-792173364303 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:39.259748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 185/365] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:17:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:39.260868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793eac9-8ac0-4830-a0a8-792173364303 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:39.314135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 181/366] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:39.315336 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed58df-9ff6-4656-b92e-ce1a5c901c10 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:39.395300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793eac9-8ac0-4830-a0a8-792173364303 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:39.411758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:39.439429 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c1e99-ccf2-46bf-a9a4-851d294ef884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:39.440158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c1e99-ccf2-46bf-a9a4-851d294ef884 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:39.440502 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c1e99-ccf2-46bf-a9a4-851d294ef884 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:39.440928 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c1e99-ccf2-46bf-a9a4-851d294ef884 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:39.450305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed58df-9ff6-4656-b92e-ce1a5c901c10 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:39.452928 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c1e99-ccf2-46bf-a9a4-851d294ef884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:39.459984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-538c1e99-ccf2-46bf-a9a4-851d294ef884 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:17:39.459984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:39.459984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:39.459984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:39.459984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:39.459984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:39.459984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:39.459984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:39.459984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:39.459984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:39.459984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:39.459984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:39.459984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:39.459984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:39.459984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:39.459984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:39.461149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:39.461149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:39.461149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:39.461149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:39.461149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:39.461149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:39.461149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:39.461149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:39.461149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:39.461149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:39.461149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:39.461149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:39.461149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:39.461149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:39.461149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:39.461149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:39.462244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:39.462244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:39.462244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:39.462244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:39.462244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:39.462244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:39.462244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:39.462244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:39.462244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:39.462244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:39.462244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:17:39.462244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:39.465122 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 186/367] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:17:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:39.467140 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c1e99-ccf2-46bf-a9a4-851d294ef884 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:39.477618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:39.495879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05628cb-7b6f-4f74-9032-91e574a03f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:39.497000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05628cb-7b6f-4f74-9032-91e574a03f1e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:39.497600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05628cb-7b6f-4f74-9032-91e574a03f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:39.497893 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05628cb-7b6f-4f74-9032-91e574a03f1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:39.498653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05628cb-7b6f-4f74-9032-91e574a03f1e None admin] RBAC: Adding query filter params (group.id=4d2c82261800420081b2b0721af10b12, role.id=e4309dcb11f74e10ba108b7b00264476, scope.project.id=8959cf36bec74b8ca22049270ee47cb2) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:39.499144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05628cb-7b6f-4f74-9032-91e574a03f1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:39.502160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05628cb-7b6f-4f74-9032-91e574a03f1e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:39.510813 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 182/368] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 19 10:17:39 2022] GET /identity/v3/role_assignments?group.id=4d2c82261800420081b2b0721af10b12&scope.project.id=8959cf36bec74b8ca22049270ee47cb2&role.id=e4309dcb11f74e10ba108b7b00264476 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:39.511872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05628cb-7b6f-4f74-9032-91e574a03f1e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:39.602383 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c1e99-ccf2-46bf-a9a4-851d294ef884 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:39.655298 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e21f0-2f66-4446-85d4-6acdaa16320b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:39.655987 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e21f0-2f66-4446-85d4-6acdaa16320b None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:39.656252 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e21f0-2f66-4446-85d4-6acdaa16320b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:39.656531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e21f0-2f66-4446-85d4-6acdaa16320b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:39.663003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05628cb-7b6f-4f74-9032-91e574a03f1e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:39.711097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60ea71-fb9a-4bfb-955c-c66f9de1cbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:39.712104 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60ea71-fb9a-4bfb-955c-c66f9de1cbbd None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:39.712283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:39.713372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60ea71-fb9a-4bfb-955c-c66f9de1cbbd None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:39.713793 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60ea71-fb9a-4bfb-955c-c66f9de1cbbd None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:39.717035 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60ea71-fb9a-4bfb-955c-c66f9de1cbbd None admin] RBAC: Authorizing `identity:get_project(project_id=a52b1cb46c36451498d5787d735e4f12)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:39.718493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60ea71-fb9a-4bfb-955c-c66f9de1cbbd None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:39.721254 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 183/369] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:39 2022] GET /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:39.722364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60ea71-fb9a-4bfb-955c-c66f9de1cbbd None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:39.766919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e21f0-2f66-4446-85d4-6acdaa16320b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:39.768356 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e21f0-2f66-4446-85d4-6acdaa16320b None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:39.800849 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:39.869912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60ea71-fb9a-4bfb-955c-c66f9de1cbbd None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:39.888068 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 187/370] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:39.893966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e21f0-2f66-4446-85d4-6acdaa16320b None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:39.921914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddd058-8891-4a43-86ef-e992313865b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:39.925778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddd058-8891-4a43-86ef-e992313865b7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:39.926314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddd058-8891-4a43-86ef-e992313865b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:39.926800 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddd058-8891-4a43-86ef-e992313865b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:39.927772 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddd058-8891-4a43-86ef-e992313865b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:39.928370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddd058-8891-4a43-86ef-e992313865b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:39.930009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddd058-8891-4a43-86ef-e992313865b7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:39.941782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 184/371] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:39.942936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddd058-8891-4a43-86ef-e992313865b7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:39.980283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:40.013916 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e21f0-2f66-4446-85d4-6acdaa16320b None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:40.050617 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318809cd-9929-4af7-bd91-a872da5ecfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:40.051607 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318809cd-9929-4af7-bd91-a872da5ecfed None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:40.052087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318809cd-9929-4af7-bd91-a872da5ecfed None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:40.052549 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318809cd-9929-4af7-bd91-a872da5ecfed None admin] PATH_INFO: `/v3/groups/4d2c82261800420081b2b0721af10b12` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:40.053913 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318809cd-9929-4af7-bd91-a872da5ecfed None admin] RBAC: Authorizing `identity:get_group(group_id=4d2c82261800420081b2b0721af10b12)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:40.055563 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318809cd-9929-4af7-bd91-a872da5ecfed None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:40.057713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 188/372] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 19 10:17:39 2022] GET /identity/v3/groups/4d2c82261800420081b2b0721af10b12 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:40.058676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318809cd-9929-4af7-bd91-a872da5ecfed None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:40.076349 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddd058-8891-4a43-86ef-e992313865b7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:40.090321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:40.117887 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263df063-6d47-4c1d-83a1-c33b75fa5712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:40.118959 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263df063-6d47-4c1d-83a1-c33b75fa5712 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:40.119459 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263df063-6d47-4c1d-83a1-c33b75fa5712 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:40.119926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263df063-6d47-4c1d-83a1-c33b75fa5712 None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:40.121404 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263df063-6d47-4c1d-83a1-c33b75fa5712 None admin] RBAC: Authorizing `identity:get_project(project_id=a52b1cb46c36451498d5787d735e4f12)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:40.123070 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263df063-6d47-4c1d-83a1-c33b75fa5712 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:40.124892 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 185/373] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:39 2022] GET /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:40.126020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263df063-6d47-4c1d-83a1-c33b75fa5712 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:40.203531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318809cd-9929-4af7-bd91-a872da5ecfed None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:40.245399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc27020-a5b4-4575-adcf-e4c5f0d40177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:40.245902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263df063-6d47-4c1d-83a1-c33b75fa5712 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:40.246594 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc27020-a5b4-4575-adcf-e4c5f0d40177 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:40.247062 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc27020-a5b4-4575-adcf-e4c5f0d40177 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:40.247871 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc27020-a5b4-4575-adcf-e4c5f0d40177 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:40.260862 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:40.298565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20307628-07a9-4cbf-a7d8-39a53f1422fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:40.299683 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20307628-07a9-4cbf-a7d8-39a53f1422fd None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:40.300024 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20307628-07a9-4cbf-a7d8-39a53f1422fd None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:40.300368 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20307628-07a9-4cbf-a7d8-39a53f1422fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:40.301391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20307628-07a9-4cbf-a7d8-39a53f1422fd None admin] RBAC: Adding query filter params (group.id=4d2c82261800420081b2b0721af10b12, role.id=c44d082d88894b888e94797acf7655b7, scope.project.id=a52b1cb46c36451498d5787d735e4f12) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:40.301711 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20307628-07a9-4cbf-a7d8-39a53f1422fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:40.303315 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20307628-07a9-4cbf-a7d8-39a53f1422fd None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:40.309409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 186/374] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 19 10:17:40 2022] GET /identity/v3/role_assignments?group.id=4d2c82261800420081b2b0721af10b12&scope.project.id=a52b1cb46c36451498d5787d735e4f12&role.id=c44d082d88894b888e94797acf7655b7 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:40.317095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20307628-07a9-4cbf-a7d8-39a53f1422fd None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:40.383421 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc27020-a5b4-4575-adcf-e4c5f0d40177 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:40.384937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc27020-a5b4-4575-adcf-e4c5f0d40177 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:40.405436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:40.473178 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20307628-07a9-4cbf-a7d8-39a53f1422fd None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:40.515186 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a8dce-60b3-47b9-8468-540575ce5183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:40.515875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a8dce-60b3-47b9-8468-540575ce5183 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:40.516091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a8dce-60b3-47b9-8468-540575ce5183 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:40.516359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a8dce-60b3-47b9-8468-540575ce5183 None admin] PATH_INFO: `/v3/groups/4d2c82261800420081b2b0721af10b12` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:40.517584 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a8dce-60b3-47b9-8468-540575ce5183 None admin] RBAC: Authorizing `identity:get_group(group_id=4d2c82261800420081b2b0721af10b12)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:40.519004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a8dce-60b3-47b9-8468-540575ce5183 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:40.521960 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 187/375] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 19 10:17:40 2022] GET /identity/v3/groups/4d2c82261800420081b2b0721af10b12 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:40.527470 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a8dce-60b3-47b9-8468-540575ce5183 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:40.581508 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 189/376] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:40.587351 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc27020-a5b4-4575-adcf-e4c5f0d40177 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:40.642884 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:40.693757 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a8dce-60b3-47b9-8468-540575ce5183 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:40.713258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:40.730131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc27020-a5b4-4575-adcf-e4c5f0d40177 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:40.772504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809a326-4750-46ba-829c-34bc30bea424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:40.773682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809a326-4750-46ba-829c-34bc30bea424 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:40.774241 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809a326-4750-46ba-829c-34bc30bea424 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:40.774973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809a326-4750-46ba-829c-34bc30bea424 None admin] PATH_INFO: `/v3/projects/1e8dac977aca46d48c891cd4c281d8cb` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:40.776296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809a326-4750-46ba-829c-34bc30bea424 None admin] RBAC: Authorizing `identity:get_project(project_id=1e8dac977aca46d48c891cd4c281d8cb)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:40.778091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809a326-4750-46ba-829c-34bc30bea424 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:40.780895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 190/377] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:40 2022] GET /identity/v3/projects/1e8dac977aca46d48c891cd4c281d8cb => generated 344 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:40.791312 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9b24d-38ad-4bf9-b849-9f9edc166df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:40.791480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809a326-4750-46ba-829c-34bc30bea424 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:40.791560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9b24d-38ad-4bf9-b849-9f9edc166df2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:40.791560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9b24d-38ad-4bf9-b849-9f9edc166df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:40.791560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9b24d-38ad-4bf9-b849-9f9edc166df2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:40.791560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c9b24d-38ad-4bf9-b849-9f9edc166df2 None admin] RBAC: Adding query filter params (group.id=4d2c82261800420081b2b0721af10b12, role.id=e4309dcb11f74e10ba108b7b00264476, scope.project.id=a52b1cb46c36451498d5787d735e4f12) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:40.791560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c9b24d-38ad-4bf9-b849-9f9edc166df2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:40.791560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c9b24d-38ad-4bf9-b849-9f9edc166df2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:40.797554 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 188/378] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 19 10:17:40 2022] GET /identity/v3/role_assignments?group.id=4d2c82261800420081b2b0721af10b12&scope.project.id=a52b1cb46c36451498d5787d735e4f12&role.id=e4309dcb11f74e10ba108b7b00264476 => generated 260 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:40.984571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809a326-4750-46ba-829c-34bc30bea424 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:40.998712 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:41.029474 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db28bd-2b37-40cd-b279-b96232092d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:41.030112 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db28bd-2b37-40cd-b279-b96232092d3f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:41.030393 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db28bd-2b37-40cd-b279-b96232092d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:41.030667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db28bd-2b37-40cd-b279-b96232092d3f None admin] PATH_INFO: `/v3/groups/944aa72aaa2649c8837445758b5f3b24` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:41.035335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d2070-eba6-4557-9d43-5b13a4ebe5c1 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:41.035892 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d2070-eba6-4557-9d43-5b13a4ebe5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:41.036374 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d2070-eba6-4557-9d43-5b13a4ebe5c1 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:41.037894 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db28bd-2b37-40cd-b279-b96232092d3f None admin] RBAC: Authorizing `identity:get_group(group_id=944aa72aaa2649c8837445758b5f3b24)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:41.038728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 189/379] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:41.041258 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db28bd-2b37-40cd-b279-b96232092d3f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:41.042514 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 191/380] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 19 10:17:40 2022] GET /identity/v3/groups/944aa72aaa2649c8837445758b5f3b24 => generated 229 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:41.045093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e18fb-9387-435f-b8c9-1f87f414e18f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:41.045755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e18fb-9387-435f-b8c9-1f87f414e18f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:41.046239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e18fb-9387-435f-b8c9-1f87f414e18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:41.048205 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db28bd-2b37-40cd-b279-b96232092d3f None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:41.071532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6b6e18fb-9387-435f-b8c9-1f87f414e18f None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:41.073617 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6b6e18fb-9387-435f-b8c9-1f87f414e18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:41.160834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 190/381] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:41.168363 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db28bd-2b37-40cd-b279-b96232092d3f None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:41.202798 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd82be-fdb2-40d3-bdf9-a0d7abcdcfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:41.203420 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd82be-fdb2-40d3-bdf9-a0d7abcdcfa2 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:41.203702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd82be-fdb2-40d3-bdf9-a0d7abcdcfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:41.203974 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd82be-fdb2-40d3-bdf9-a0d7abcdcfa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:41.204852 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd82be-fdb2-40d3-bdf9-a0d7abcdcfa2 None admin] RBAC: Adding query filter params (group.id=944aa72aaa2649c8837445758b5f3b24, role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.project.id=1e8dac977aca46d48c891cd4c281d8cb) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:41.205211 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd82be-fdb2-40d3-bdf9-a0d7abcdcfa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:41.206655 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd82be-fdb2-40d3-bdf9-a0d7abcdcfa2 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:41.212604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 192/382] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 19 10:17:41 2022] GET /identity/v3/role_assignments?group.id=944aa72aaa2649c8837445758b5f3b24&scope.project.id=1e8dac977aca46d48c891cd4c281d8cb&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:41.695722 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e18fb-9387-435f-b8c9-1f87f414e18f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:41.833436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e18fb-9387-435f-b8c9-1f87f414e18f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:41.841775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:41.869824 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191a0d3-048d-4ce9-aad7-8d280a55cca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:41.870771 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191a0d3-048d-4ce9-aad7-8d280a55cca1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:41.871218 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191a0d3-048d-4ce9-aad7-8d280a55cca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:41.871789 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191a0d3-048d-4ce9-aad7-8d280a55cca1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:41.883064 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191a0d3-048d-4ce9-aad7-8d280a55cca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:41.888796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-3191a0d3-048d-4ce9-aad7-8d280a55cca1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:41.888796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:41.888796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:41.888796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:41.888796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:41.888796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:41.888796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:41.888796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:41.888796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:41.888796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:41.888796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:41.888796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:41.888796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:41.888796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:41.888796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:41.888796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:41.889801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:41.889801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:41.889801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:41.889801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:41.889801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:41.889801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:41.889801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:41.889801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:41.889801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:41.889801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:41.889801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:41.889801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:41.889801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:41.889801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:41.889801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:41.889801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:41.890608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:41.890608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:41.890608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:41.890608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:41.890608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:41.890608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:41.890608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:41.890608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:41.890608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:41.890608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:41.890608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:41.890608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:41.894366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 191/383] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:41.900253 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd82be-fdb2-40d3-bdf9-a0d7abcdcfa2 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:41.959653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:42.043452 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd82be-fdb2-40d3-bdf9-a0d7abcdcfa2 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:42.082737 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007f6f3-eb92-41de-bcfa-46d3e4e7632c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:42.083344 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007f6f3-eb92-41de-bcfa-46d3e4e7632c None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:42.083617 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007f6f3-eb92-41de-bcfa-46d3e4e7632c None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:42.083885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007f6f3-eb92-41de-bcfa-46d3e4e7632c None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:42.084534 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007f6f3-eb92-41de-bcfa-46d3e4e7632c None admin] RBAC: Adding query filter params (name=member) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:42.084829 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007f6f3-eb92-41de-bcfa-46d3e4e7632c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:42.086236 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007f6f3-eb92-41de-bcfa-46d3e4e7632c None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:42.101973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 193/384] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:42.108176 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191a0d3-048d-4ce9-aad7-8d280a55cca1 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:42.239977 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06bbca3-7873-4b82-8ac9-1faf7dc2f30a None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:42.240160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06bbca3-7873-4b82-8ac9-1faf7dc2f30a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:42.240440 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06bbca3-7873-4b82-8ac9-1faf7dc2f30a None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:42.242230 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 194/385] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:42.247249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47f975-c0ce-4cea-8f7b-a46282a95c98 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:42.248072 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47f975-c0ce-4cea-8f7b-a46282a95c98 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:42.248358 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47f975-c0ce-4cea-8f7b-a46282a95c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:42.279309 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-1c47f975-c0ce-4cea-8f7b-a46282a95c98 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:42.280516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-1c47f975-c0ce-4cea-8f7b-a46282a95c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:42.298589 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191a0d3-048d-4ce9-aad7-8d280a55cca1 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:42.363339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984ed0a-fab9-4e58-b257-89a5b80650ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:42.363974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984ed0a-fab9-4e58-b257-89a5b80650ef None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:42.364255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984ed0a-fab9-4e58-b257-89a5b80650ef None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:42.364530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984ed0a-fab9-4e58-b257-89a5b80650ef None admin] PATH_INFO: `/v3/groups/4d2c82261800420081b2b0721af10b12` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:42.365821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984ed0a-fab9-4e58-b257-89a5b80650ef None admin] RBAC: Authorizing `identity:get_group(group_id=4d2c82261800420081b2b0721af10b12)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:42.367245 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984ed0a-fab9-4e58-b257-89a5b80650ef None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:42.368806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 192/386] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 19 10:17:42 2022] GET /identity/v3/groups/4d2c82261800420081b2b0721af10b12 => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:42.377493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984ed0a-fab9-4e58-b257-89a5b80650ef None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:42.402611 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 195/387] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:17:42.435309 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:42.522361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984ed0a-fab9-4e58-b257-89a5b80650ef None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:42.569131 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60621fd7-ffd9-4c8e-9268-bd72d62b08d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:42.570099 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60621fd7-ffd9-4c8e-9268-bd72d62b08d3 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:42.570604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60621fd7-ffd9-4c8e-9268-bd72d62b08d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:42.571103 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60621fd7-ffd9-4c8e-9268-bd72d62b08d3 None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:42.572437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60621fd7-ffd9-4c8e-9268-bd72d62b08d3 None admin] RBAC: Authorizing `identity:get_project(project_id=8959cf36bec74b8ca22049270ee47cb2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:42.578130 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60621fd7-ffd9-4c8e-9268-bd72d62b08d3 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:42.580307 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 193/388] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:42 2022] GET /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:42.585785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47f975-c0ce-4cea-8f7b-a46282a95c98 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:42.630717 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:42.729011 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47f975-c0ce-4cea-8f7b-a46282a95c98 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:42.764771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe03ea-8c65-45e8-8714-1a6438d180aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:42.765455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe03ea-8c65-45e8-8714-1a6438d180aa None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:42.765732 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe03ea-8c65-45e8-8714-1a6438d180aa None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:42.766000 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe03ea-8c65-45e8-8714-1a6438d180aa None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2/groups/4d2c82261800420081b2b0721af10b12/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:42.767950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe03ea-8c65-45e8-8714-1a6438d180aa None admin] RBAC: Authorizing `identity:create_grant(project_id=8959cf36bec74b8ca22049270ee47cb2, group_id=4d2c82261800420081b2b0721af10b12, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:42.769415 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe03ea-8c65-45e8-8714-1a6438d180aa None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:42.791849 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60621fd7-ffd9-4c8e-9268-bd72d62b08d3 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:42.881176 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 196/389] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Sep 19 10:17:42 2022] PUT /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2/groups/4d2c82261800420081b2b0721af10b12/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:17:42.907037 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60621fd7-ffd9-4c8e-9268-bd72d62b08d3 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:42.908771 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:42.944164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bb857-0df0-4fb6-8840-c3d6476bcae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:42.944816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bb857-0df0-4fb6-8840-c3d6476bcae1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:42.945198 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bb857-0df0-4fb6-8840-c3d6476bcae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:42.945484 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bb857-0df0-4fb6-8840-c3d6476bcae1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:42.957278 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846bb857-0df0-4fb6-8840-c3d6476bcae1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:42.962466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-846bb857-0df0-4fb6-8840-c3d6476bcae1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:42.962466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:42.962466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:42.962466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:42.962466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:42.962466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:42.962466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:42.962466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:42.962466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:42.962466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:42.962466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:42.962466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:42.962466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:42.962466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:42.962466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:42.962466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:42.963396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:42.963396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:42.963396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:42.963396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:42.963396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:42.963396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:42.963396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:42.963396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:42.963396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:42.963396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:42.963396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:42.963396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:42.963396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:42.963396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:42.963396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:42.963396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:42.964210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:42.964210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:42.964210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:42.964210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:42.964210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:42.964210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:42.964210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:42.964210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:42.964210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:42.964210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:42.964210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:42.964210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:42.967383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 194/390] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:17:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:17:42.974217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe03ea-8c65-45e8-8714-1a6438d180aa None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:43.106395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:43.122487 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe03ea-8c65-45e8-8714-1a6438d180aa None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:43.171253 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73710b76-d38b-4d81-8c3f-ecf272bcaaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:43.171945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73710b76-d38b-4d81-8c3f-ecf272bcaaf6 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:43.173638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73710b76-d38b-4d81-8c3f-ecf272bcaaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:43.173919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73710b76-d38b-4d81-8c3f-ecf272bcaaf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:43.174614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73710b76-d38b-4d81-8c3f-ecf272bcaaf6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:43.174916 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73710b76-d38b-4d81-8c3f-ecf272bcaaf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:43.176310 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73710b76-d38b-4d81-8c3f-ecf272bcaaf6 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:43.193565 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 197/391] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:17:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:43.198996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bb857-0df0-4fb6-8840-c3d6476bcae1 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:43.322094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bb857-0df0-4fb6-8840-c3d6476bcae1 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:43.362163 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846aa42d-3cf0-4646-a861-c925d78ff096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:43.362743 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846aa42d-3cf0-4646-a861-c925d78ff096 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:43.363021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846aa42d-3cf0-4646-a861-c925d78ff096 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:43.363301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846aa42d-3cf0-4646-a861-c925d78ff096 None admin] PATH_INFO: `/v3/groups/4d2c82261800420081b2b0721af10b12` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:43.364568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846aa42d-3cf0-4646-a861-c925d78ff096 None admin] RBAC: Authorizing `identity:get_group(group_id=4d2c82261800420081b2b0721af10b12)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:43.366097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846aa42d-3cf0-4646-a861-c925d78ff096 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:43.373179 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 195/392] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 19 10:17:43 2022] GET /identity/v3/groups/4d2c82261800420081b2b0721af10b12 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:43.378270 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73710b76-d38b-4d81-8c3f-ecf272bcaaf6 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:43.493924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f71cd-ff56-45c6-acc1-ab62c9b273c2 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:43.494995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f71cd-ff56-45c6-acc1-ab62c9b273c2 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:43.495466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f71cd-ff56-45c6-acc1-ab62c9b273c2 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:43.501167 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 196/393] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:17:43.503437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ce6a9-6071-43fd-a7db-6a30083dc150 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:43.503914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ce6a9-6071-43fd-a7db-6a30083dc150 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:43.504504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ce6a9-6071-43fd-a7db-6a30083dc150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:43.527502 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73710b76-d38b-4d81-8c3f-ecf272bcaaf6 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:43.540595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-9d3ce6a9-6071-43fd-a7db-6a30083dc150 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:43.541814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9d3ce6a9-6071-43fd-a7db-6a30083dc150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:43.572952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fefd9-3d49-4957-9e10-ac1efd54d798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:43.573881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fefd9-3d49-4957-9e10-ac1efd54d798 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:43.574317 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fefd9-3d49-4957-9e10-ac1efd54d798 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:43.574731 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fefd9-3d49-4957-9e10-ac1efd54d798 None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:43.576079 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fefd9-3d49-4957-9e10-ac1efd54d798 None admin] RBAC: Authorizing `identity:get_project(project_id=8959cf36bec74b8ca22049270ee47cb2)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:43.577705 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fefd9-3d49-4957-9e10-ac1efd54d798 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:43.578418 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:43.579972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 198/394] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:43 2022] GET /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:43.581285 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b12a0-5997-4649-ac07-acd9f8072083 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:43.581720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b12a0-5997-4649-ac07-acd9f8072083 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:43.582136 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b12a0-5997-4649-ac07-acd9f8072083 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:43.583468 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 199/395] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:43.587886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b12a0-5997-4649-ac07-acd9f8072083 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:43.614997 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:43.630192 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 197/396] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:17:43.631810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decb207-58e7-4c12-997c-7b68e2d129a6 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:43.632303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decb207-58e7-4c12-997c-7b68e2d129a6 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:43.632787 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decb207-58e7-4c12-997c-7b68e2d129a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:43.677913 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6decb207-58e7-4c12-997c-7b68e2d129a6 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:43.679439 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6decb207-58e7-4c12-997c-7b68e2d129a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:43.768055 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b12a0-5997-4649-ac07-acd9f8072083 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:43.807383 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56371b-a482-417a-953c-bd13a529b57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:43.808506 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56371b-a482-417a-953c-bd13a529b57d None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:43.809004 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56371b-a482-417a-953c-bd13a529b57d None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:43.810030 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56371b-a482-417a-953c-bd13a529b57d None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2/groups/4d2c82261800420081b2b0721af10b12/roles/e4309dcb11f74e10ba108b7b00264476` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:43.812671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56371b-a482-417a-953c-bd13a529b57d None admin] RBAC: Authorizing `identity:create_grant(project_id=8959cf36bec74b8ca22049270ee47cb2, group_id=4d2c82261800420081b2b0721af10b12, role_id=e4309dcb11f74e10ba108b7b00264476)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:43.814557 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56371b-a482-417a-953c-bd13a529b57d None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:43.827704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 198/397] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:43.859197 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb278f-bea9-48aa-bfe5-ffe5c18e0b80 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:43.859524 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb278f-bea9-48aa-bfe5-ffe5c18e0b80 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:43.859762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb278f-bea9-48aa-bfe5-ffe5c18e0b80 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:43.861416 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 199/398] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:43.866332 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e443a17-699b-45ef-9bd5-0451ab010d0c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:43.866620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e443a17-699b-45ef-9bd5-0451ab010d0c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:43.866898 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e443a17-699b-45ef-9bd5-0451ab010d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:43.887316 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5e443a17-699b-45ef-9bd5-0451ab010d0c None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:43.888653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5e443a17-699b-45ef-9bd5-0451ab010d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:43.900186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 200/399] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Sep 19 10:17:43 2022] PUT /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2/groups/4d2c82261800420081b2b0721af10b12/roles/e4309dcb11f74e10ba108b7b00264476 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:17:43.941775 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56371b-a482-417a-953c-bd13a529b57d None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:43.984746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 200/400] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:44.059506 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:44.085526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56371b-a482-417a-953c-bd13a529b57d None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:44.130222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48bdc0-13fc-4d21-bf2f-bce8ad21df44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:44.130222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48bdc0-13fc-4d21-bf2f-bce8ad21df44 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:44.130222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48bdc0-13fc-4d21-bf2f-bce8ad21df44 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:44.130222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48bdc0-13fc-4d21-bf2f-bce8ad21df44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:44.136103 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48bdc0-13fc-4d21-bf2f-bce8ad21df44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:44.142204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-0b48bdc0-13fc-4d21-bf2f-bce8ad21df44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:44.142204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:44.142204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:44.142204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:44.142204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:44.142204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:44.142204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:44.142204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:44.142204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:44.142204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:44.142204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:44.142204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:44.142204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:44.142204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:44.142204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:44.142204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:44.143177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:44.143177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:44.143177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:44.143177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:44.143177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:44.143177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:44.143177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:44.143177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:44.143177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:44.143177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:44.143177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:44.143177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:44.143177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:44.143177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:44.143177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:44.143177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:44.144033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:44.144033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:44.144033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:44.144033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:44.144033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:44.144033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:44.144033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:44.144033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:44.144033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:44.144033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:44.144033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:44.144033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:44.144033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 201/401] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:17:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:44.149831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e443a17-699b-45ef-9bd5-0451ab010d0c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:44.174373 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:44.218049 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48bdc0-13fc-4d21-bf2f-bce8ad21df44 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:44.283864 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e443a17-699b-45ef-9bd5-0451ab010d0c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:44.340995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788dbd0e-c5f0-4e17-9619-db9f7e5a25f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:44.341662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788dbd0e-c5f0-4e17-9619-db9f7e5a25f4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:44.341941 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788dbd0e-c5f0-4e17-9619-db9f7e5a25f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:44.342575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788dbd0e-c5f0-4e17-9619-db9f7e5a25f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:44.343254 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788dbd0e-c5f0-4e17-9619-db9f7e5a25f4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:44.343890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788dbd0e-c5f0-4e17-9619-db9f7e5a25f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:44.345545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788dbd0e-c5f0-4e17-9619-db9f7e5a25f4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:44.355143 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 201/402] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:17:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:44.355964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788dbd0e-c5f0-4e17-9619-db9f7e5a25f4 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:44.371304 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48bdc0-13fc-4d21-bf2f-bce8ad21df44 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:44.409658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8ee2e-7862-49f0-86e2-97cea48d8278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:44.410452 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8ee2e-7862-49f0-86e2-97cea48d8278 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:44.410732 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8ee2e-7862-49f0-86e2-97cea48d8278 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:44.411006 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8ee2e-7862-49f0-86e2-97cea48d8278 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:44.414013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:44.424212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8ee2e-7862-49f0-86e2-97cea48d8278 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:44.429800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-acc8ee2e-7862-49f0-86e2-97cea48d8278 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:44.429800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:44.429800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:44.429800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:44.429800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:44.429800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:44.429800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:44.429800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:44.429800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:44.429800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:44.429800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:44.429800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:44.429800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:44.429800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:44.429800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:44.429800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:44.432279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:44.432279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:44.432279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:44.432279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:44.432279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:44.432279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:44.432279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:44.432279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:44.432279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:44.432279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:44.432279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:44.432279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:44.432279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:44.432279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:44.432279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:44.432279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:44.433340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:44.433340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:44.433340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:44.433340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:44.433340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:44.433340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:44.433340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:44.433340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:44.433340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:44.433340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:44.433340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:44.433340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:44.434160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 202/403] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:17:44.435352 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8ee2e-7862-49f0-86e2-97cea48d8278 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:44.449152 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:44.470155 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788dbd0e-c5f0-4e17-9619-db9f7e5a25f4 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:44.512338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a49247-00e0-4cca-9b18-8c689cf136c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:44.513806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a49247-00e0-4cca-9b18-8c689cf136c6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:44.514285 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a49247-00e0-4cca-9b18-8c689cf136c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:44.514856 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a49247-00e0-4cca-9b18-8c689cf136c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:44.528339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a49247-00e0-4cca-9b18-8c689cf136c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:44.534989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-86a49247-00e0-4cca-9b18-8c689cf136c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:17:44.534989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:44.534989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:44.534989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:44.534989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:44.534989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:44.534989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:44.534989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:44.534989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:44.534989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:44.534989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:44.534989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:44.534989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:44.534989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:44.534989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:44.534989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:44.535875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:44.535875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:44.535875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:44.535875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:44.535875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:44.535875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:44.535875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:44.535875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:44.535875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:44.535875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:44.535875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:44.535875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:44.535875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:44.535875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:44.535875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:44.535875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:44.536691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:44.536691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:44.536691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:44.536691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:44.536691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:44.536691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:44.536691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:44.536691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:44.536691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:44.536691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:44.536691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:17:44.536691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:44.541389 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 202/404] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:17:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:44.541389 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a49247-00e0-4cca-9b18-8c689cf136c6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:44.558188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8ee2e-7862-49f0-86e2-97cea48d8278 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:44.597963 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4e29b-bfce-4745-b1a0-296507cc7df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:44.599144 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4e29b-bfce-4745-b1a0-296507cc7df4 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:44.599528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4e29b-bfce-4745-b1a0-296507cc7df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:44.599906 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4e29b-bfce-4745-b1a0-296507cc7df4 None admin] PATH_INFO: `/v3/groups/4d2c82261800420081b2b0721af10b12` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:44.601189 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4e29b-bfce-4745-b1a0-296507cc7df4 None admin] RBAC: Authorizing `identity:get_group(group_id=4d2c82261800420081b2b0721af10b12)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:44.603473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4e29b-bfce-4745-b1a0-296507cc7df4 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:44.606514 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 203/405] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 19 10:17:44 2022] GET /identity/v3/groups/4d2c82261800420081b2b0721af10b12 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:44.608270 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4e29b-bfce-4745-b1a0-296507cc7df4 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:44.657390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a49247-00e0-4cca-9b18-8c689cf136c6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:44.682291 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:44.697297 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58066bc5-bcad-4015-9cd6-7b280e998fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:44.698570 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58066bc5-bcad-4015-9cd6-7b280e998fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:44.698999 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58066bc5-bcad-4015-9cd6-7b280e998fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:44.699429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58066bc5-bcad-4015-9cd6-7b280e998fd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:44.701151 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58066bc5-bcad-4015-9cd6-7b280e998fd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:44.701622 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58066bc5-bcad-4015-9cd6-7b280e998fd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:44.703774 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58066bc5-bcad-4015-9cd6-7b280e998fd4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:44.713617 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 203/406] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:44.714676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58066bc5-bcad-4015-9cd6-7b280e998fd4 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:44.720271 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:44.731072 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4e29b-bfce-4745-b1a0-296507cc7df4 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:44.768508 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ad15-5a0a-4dd6-9bec-cf591a5a8dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:44.770056 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ad15-5a0a-4dd6-9bec-cf591a5a8dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:44.770748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ad15-5a0a-4dd6-9bec-cf591a5a8dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:44.771340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ad15-5a0a-4dd6-9bec-cf591a5a8dc6 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:44.772445 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140ad15-5a0a-4dd6-9bec-cf591a5a8dc6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:44.773282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140ad15-5a0a-4dd6-9bec-cf591a5a8dc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:44.775149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140ad15-5a0a-4dd6-9bec-cf591a5a8dc6 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:44.789131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 204/407] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:44.790317 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ad15-5a0a-4dd6-9bec-cf591a5a8dc6 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:44.836284 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58066bc5-bcad-4015-9cd6-7b280e998fd4 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:44.878624 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8351374-717e-4866-bb15-04dc1604a5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:44.879514 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8351374-717e-4866-bb15-04dc1604a5fa None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:44.879953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8351374-717e-4866-bb15-04dc1604a5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:44.880382 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8351374-717e-4866-bb15-04dc1604a5fa None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:44.881776 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8351374-717e-4866-bb15-04dc1604a5fa None admin] RBAC: Authorizing `identity:get_project(project_id=a52b1cb46c36451498d5787d735e4f12)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:44.883366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8351374-717e-4866-bb15-04dc1604a5fa None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:44.885484 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 204/408] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:44 2022] GET /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:44.886417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8351374-717e-4866-bb15-04dc1604a5fa None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:44.914784 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ad15-5a0a-4dd6-9bec-cf591a5a8dc6 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:44.949629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:44.962318 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0bf5d8-152b-4bd9-82f7-d7ab640bd4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:44.963804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bf5d8-152b-4bd9-82f7-d7ab640bd4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:44.964020 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bf5d8-152b-4bd9-82f7-d7ab640bd4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:44.964462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bf5d8-152b-4bd9-82f7-d7ab640bd4c7 None admin] PATH_INFO: `/v3/groups/4d2c82261800420081b2b0721af10b12` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:44.967243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0bf5d8-152b-4bd9-82f7-d7ab640bd4c7 None admin] RBAC: Authorizing `identity:get_group(group_id=4d2c82261800420081b2b0721af10b12)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:44.969607 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0bf5d8-152b-4bd9-82f7-d7ab640bd4c7 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:44.971592 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:44.973560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 205/409] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 19 10:17:44 2022] GET /identity/v3/groups/4d2c82261800420081b2b0721af10b12 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:44.979160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0bf5d8-152b-4bd9-82f7-d7ab640bd4c7 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:45.008377 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8351374-717e-4866-bb15-04dc1604a5fa None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:45.059275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126767b6-ce1f-4de0-9da6-030891df336f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:45.060093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126767b6-ce1f-4de0-9da6-030891df336f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:45.060468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126767b6-ce1f-4de0-9da6-030891df336f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:45.060842 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126767b6-ce1f-4de0-9da6-030891df336f None admin] PATH_INFO: `/v3/groups/944aa72aaa2649c8837445758b5f3b24` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:45.062256 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126767b6-ce1f-4de0-9da6-030891df336f None admin] RBAC: Authorizing `identity:get_group(group_id=944aa72aaa2649c8837445758b5f3b24)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:45.063829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126767b6-ce1f-4de0-9da6-030891df336f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:45.065544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 205/410] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 19 10:17:44 2022] GET /identity/v3/groups/944aa72aaa2649c8837445758b5f3b24 => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:45.069944 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126767b6-ce1f-4de0-9da6-030891df336f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:45.123112 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0bf5d8-152b-4bd9-82f7-d7ab640bd4c7 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:45.174347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a236c-69c2-460d-b8b4-f01d647bb521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:45.175008 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a236c-69c2-460d-b8b4-f01d647bb521 None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:45.175285 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a236c-69c2-460d-b8b4-f01d647bb521 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:45.175556 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a236c-69c2-460d-b8b4-f01d647bb521 None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12/groups/4d2c82261800420081b2b0721af10b12/roles/e4309dcb11f74e10ba108b7b00264476` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:45.178344 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a236c-69c2-460d-b8b4-f01d647bb521 None admin] RBAC: Authorizing `identity:create_grant(project_id=a52b1cb46c36451498d5787d735e4f12, group_id=4d2c82261800420081b2b0721af10b12, role_id=e4309dcb11f74e10ba108b7b00264476)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:45.179994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a236c-69c2-460d-b8b4-f01d647bb521 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:45.183900 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126767b6-ce1f-4de0-9da6-030891df336f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:45.219995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078443b9-20ce-4b36-9861-2a9f3d881cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:45.220890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078443b9-20ce-4b36-9861-2a9f3d881cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:45.221382 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078443b9-20ce-4b36-9861-2a9f3d881cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:45.221919 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078443b9-20ce-4b36-9861-2a9f3d881cc0 None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:45.223245 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078443b9-20ce-4b36-9861-2a9f3d881cc0 None admin] RBAC: Authorizing `identity:get_project(project_id=a52b1cb46c36451498d5787d735e4f12)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:45.224845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078443b9-20ce-4b36-9861-2a9f3d881cc0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:45.226662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 206/411] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:45 2022] GET /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:45.227586 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078443b9-20ce-4b36-9861-2a9f3d881cc0 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:45.243308 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:45.289719 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 206/412] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Sep 19 10:17:44 2022] PUT /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12/groups/4d2c82261800420081b2b0721af10b12/roles/e4309dcb11f74e10ba108b7b00264476 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:17:45.291398 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a236c-69c2-460d-b8b4-f01d647bb521 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:45.352633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078443b9-20ce-4b36-9861-2a9f3d881cc0 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:45.398432 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:45.428569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b842b4c-92e8-4834-b17e-ab2b355417e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:45.429667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b842b4c-92e8-4834-b17e-ab2b355417e2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:45.430157 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b842b4c-92e8-4834-b17e-ab2b355417e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:45.430640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b842b4c-92e8-4834-b17e-ab2b355417e2 None admin] PATH_INFO: `/v3/projects/1e8dac977aca46d48c891cd4c281d8cb` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:45.432288 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b842b4c-92e8-4834-b17e-ab2b355417e2 None admin] RBAC: Authorizing `identity:get_project(project_id=1e8dac977aca46d48c891cd4c281d8cb)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:45.434098 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b842b4c-92e8-4834-b17e-ab2b355417e2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:45.439278 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 207/413] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:45 2022] GET /identity/v3/projects/1e8dac977aca46d48c891cd4c281d8cb => generated 344 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:45.448576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b842b4c-92e8-4834-b17e-ab2b355417e2 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:45.492924 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a236c-69c2-460d-b8b4-f01d647bb521 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:45.551369 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0237ff-5d2a-46aa-9498-194a07b1828e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:45.552467 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0237ff-5d2a-46aa-9498-194a07b1828e None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:45.552981 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0237ff-5d2a-46aa-9498-194a07b1828e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:45.553495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0237ff-5d2a-46aa-9498-194a07b1828e None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12/groups/4d2c82261800420081b2b0721af10b12/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:45.555745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0237ff-5d2a-46aa-9498-194a07b1828e None admin] RBAC: Authorizing `identity:create_grant(project_id=a52b1cb46c36451498d5787d735e4f12, group_id=4d2c82261800420081b2b0721af10b12, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:45.557404 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0237ff-5d2a-46aa-9498-194a07b1828e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:45.583738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:45.607352 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b842b4c-92e8-4834-b17e-ab2b355417e2 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:45.628199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 207/414] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Sep 19 10:17:45 2022] PUT /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12/groups/4d2c82261800420081b2b0721af10b12/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:17:45.629935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54add0ff-3a0e-4082-9102-ba3c6ed0f17f None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:45.630382 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54add0ff-3a0e-4082-9102-ba3c6ed0f17f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:45.630794 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54add0ff-3a0e-4082-9102-ba3c6ed0f17f None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:45.632227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 208/415] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:45.639956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2d223-4611-4fbe-ad15-193bbbea2345 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:45.641491 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2d223-4611-4fbe-ad15-193bbbea2345 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:45.649255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2d223-4611-4fbe-ad15-193bbbea2345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:45.672576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d60a1-8c39-4753-b420-6fcb92f5b689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:45.673291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d60a1-8c39-4753-b420-6fcb92f5b689 None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:45.673570 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d60a1-8c39-4753-b420-6fcb92f5b689 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:45.673845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d60a1-8c39-4753-b420-6fcb92f5b689 None admin] PATH_INFO: `/v3/projects/1e8dac977aca46d48c891cd4c281d8cb/groups/944aa72aaa2649c8837445758b5f3b24/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:45.675598 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-97a2d223-4611-4fbe-ad15-193bbbea2345 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:45.676830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-97a2d223-4611-4fbe-ad15-193bbbea2345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:45.679765 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d60a1-8c39-4753-b420-6fcb92f5b689 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e8dac977aca46d48c891cd4c281d8cb, group_id=944aa72aaa2649c8837445758b5f3b24, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:45.681237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d60a1-8c39-4753-b420-6fcb92f5b689 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:45.764672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 208/416] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Sep 19 10:17:45 2022] PUT /identity/v3/projects/1e8dac977aca46d48c891cd4c281d8cb/groups/944aa72aaa2649c8837445758b5f3b24/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:17:45.766202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 209/417] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:46.040475 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d60a1-8c39-4753-b420-6fcb92f5b689 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:46.146881 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d60a1-8c39-4753-b420-6fcb92f5b689 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:46.197996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357e752-4e15-490b-ba9a-74f90ac79d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:46.198667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357e752-4e15-490b-ba9a-74f90ac79d7d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:46.198948 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357e752-4e15-490b-ba9a-74f90ac79d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:46.199285 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357e752-4e15-490b-ba9a-74f90ac79d7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:46.203448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:46.220668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357e752-4e15-490b-ba9a-74f90ac79d7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:46.225930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-5357e752-4e15-490b-ba9a-74f90ac79d7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:46.225930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:46.225930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:46.225930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:46.225930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:46.225930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:46.225930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:46.225930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:46.225930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:46.225930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:46.225930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:46.225930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:46.225930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:46.225930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:46.225930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:46.225930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:46.234065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:46.234065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:46.234065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:46.234065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:46.234065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:46.234065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:46.234065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:46.234065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:46.234065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:46.234065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:46.234065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:46.234065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:46.234065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:46.234065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:46.234065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:46.234065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:46.234884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:46.234884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:46.234884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:46.234884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:46.234884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:46.234884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:46.234884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:46.234884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:46.234884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:46.234884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:46.234884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:46.234884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:46.234884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 209/418] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:46.235566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2d223-4611-4fbe-ad15-193bbbea2345 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:46.307618 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:46.320867 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb89c3c-086e-451d-a8de-5eab1cd5c2b5 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:46.324694 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb89c3c-086e-451d-a8de-5eab1cd5c2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:46.324694 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb89c3c-086e-451d-a8de-5eab1cd5c2b5 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:46.324694 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 210/419] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:46.330496 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494e025-44e4-44de-b86e-b9fbadf3eca5 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:46.330777 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494e025-44e4-44de-b86e-b9fbadf3eca5 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:46.331046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494e025-44e4-44de-b86e-b9fbadf3eca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:46.359344 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7494e025-44e4-44de-b86e-b9fbadf3eca5 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:46.361033 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7494e025-44e4-44de-b86e-b9fbadf3eca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:46.391110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2d223-4611-4fbe-ad15-193bbbea2345 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:46.458253 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fc2e9-8695-49fe-860a-56a3ed126f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:46.458866 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fc2e9-8695-49fe-860a-56a3ed126f5c None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:46.459143 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fc2e9-8695-49fe-860a-56a3ed126f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:46.459416 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fc2e9-8695-49fe-860a-56a3ed126f5c None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:46.460083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5fc2e9-8695-49fe-860a-56a3ed126f5c None admin] RBAC: Adding query filter params (name=member) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:46.460383 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5fc2e9-8695-49fe-860a-56a3ed126f5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:46.461891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5fc2e9-8695-49fe-860a-56a3ed126f5c None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:46.476471 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 210/420] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:46.478882 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fc2e9-8695-49fe-860a-56a3ed126f5c None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:46.501033 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 211/421] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:46.612936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fc2e9-8695-49fe-860a-56a3ed126f5c None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:46.620785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:46.655856 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77d187-32b8-4ce7-9f8a-a38248b9bfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:46.656473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77d187-32b8-4ce7-9f8a-a38248b9bfbf None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:46.656752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77d187-32b8-4ce7-9f8a-a38248b9bfbf None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:46.657023 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77d187-32b8-4ce7-9f8a-a38248b9bfbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:46.745868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c77d187-32b8-4ce7-9f8a-a38248b9bfbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:46.747820 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c77d187-32b8-4ce7-9f8a-a38248b9bfbf None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:46.894328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 211/422] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 10:17:46.899863 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494e025-44e4-44de-b86e-b9fbadf3eca5 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:46.976646 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77d187-32b8-4ce7-9f8a-a38248b9bfbf None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:46.985819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:47.047549 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494e025-44e4-44de-b86e-b9fbadf3eca5 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:47.111424 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:47.120546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77d187-32b8-4ce7-9f8a-a38248b9bfbf None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:47.138322 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d883485-7b6f-46e1-928e-de7583be6c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:47.139026 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d883485-7b6f-46e1-928e-de7583be6c9c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:47.139310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d883485-7b6f-46e1-928e-de7583be6c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:47.139590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d883485-7b6f-46e1-928e-de7583be6c9c None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:47.145833 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d883485-7b6f-46e1-928e-de7583be6c9c None admin] RBAC: Authorizing `identity:get_project(project_id=8959cf36bec74b8ca22049270ee47cb2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:47.147230 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d883485-7b6f-46e1-928e-de7583be6c9c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:47.149199 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 212/423] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:46 2022] GET /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:47.155834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d883485-7b6f-46e1-928e-de7583be6c9c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:47.161726 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d7854-8659-44ac-9799-00fbdc332a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:47.162359 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d7854-8659-44ac-9799-00fbdc332a56 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:47.162772 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d7854-8659-44ac-9799-00fbdc332a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:47.163046 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d7854-8659-44ac-9799-00fbdc332a56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:47.176294 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d7854-8659-44ac-9799-00fbdc332a56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:47.181635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-253d7854-8659-44ac-9799-00fbdc332a56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:47.181635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:47.181635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:47.181635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:47.181635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:47.181635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:47.181635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:47.181635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:47.181635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:47.181635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:47.181635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:47.181635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:47.181635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:47.181635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:47.181635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:47.181635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:47.182531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:47.182531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:47.182531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:47.182531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:47.182531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:47.182531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:47.182531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:47.182531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:47.182531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:47.182531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:47.182531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:47.182531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:47.182531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:47.182531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:47.182531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:47.182531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:47.184290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:47.184290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:47.184290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:47.184290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:47.184290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:47.184290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:47.184290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:47.184290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:47.184290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:47.184290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:47.184290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:47.184290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:47.184290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 212/424] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:17:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:17:47.189493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d7854-8659-44ac-9799-00fbdc332a56 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:47.260996 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:47.299206 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d7854-8659-44ac-9799-00fbdc332a56 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:47.351296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765cdff-23ee-4d97-9bbe-33a041e76096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:47.351911 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765cdff-23ee-4d97-9bbe-33a041e76096 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:47.352186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765cdff-23ee-4d97-9bbe-33a041e76096 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:47.352450 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765cdff-23ee-4d97-9bbe-33a041e76096 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:47.353129 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765cdff-23ee-4d97-9bbe-33a041e76096 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:47.353430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765cdff-23ee-4d97-9bbe-33a041e76096 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:47.363207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765cdff-23ee-4d97-9bbe-33a041e76096 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:47.376432 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 213/425] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:17:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:47.393441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765cdff-23ee-4d97-9bbe-33a041e76096 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:47.399055 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d883485-7b6f-46e1-928e-de7583be6c9c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:47.436912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ea131-54dd-4ae6-bfe3-6c63a6c532f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:47.437710 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ea131-54dd-4ae6-bfe3-6c63a6c532f8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:47.437987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ea131-54dd-4ae6-bfe3-6c63a6c532f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:47.438258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ea131-54dd-4ae6-bfe3-6c63a6c532f8 None admin] PATH_INFO: `/v3/groups/4d2c82261800420081b2b0721af10b12` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:47.439455 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ea131-54dd-4ae6-bfe3-6c63a6c532f8 None admin] RBAC: Authorizing `identity:get_group(group_id=4d2c82261800420081b2b0721af10b12)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:47.441000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ea131-54dd-4ae6-bfe3-6c63a6c532f8 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:47.442975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 213/426] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 19 10:17:47 2022] GET /identity/v3/groups/4d2c82261800420081b2b0721af10b12 => generated 226 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:47.448709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ea131-54dd-4ae6-bfe3-6c63a6c532f8 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:47.520006 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:47.562050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ea131-54dd-4ae6-bfe3-6c63a6c532f8 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:47.599582 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bc6d6-5a78-4bcb-915f-215ac4ef34a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:47.600534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bc6d6-5a78-4bcb-915f-215ac4ef34a9 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:47.601005 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bc6d6-5a78-4bcb-915f-215ac4ef34a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:47.601507 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bc6d6-5a78-4bcb-915f-215ac4ef34a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:47.602750 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01bc6d6-5a78-4bcb-915f-215ac4ef34a9 None admin] RBAC: Adding query filter params (group.id=4d2c82261800420081b2b0721af10b12, role.id=c44d082d88894b888e94797acf7655b7, scope.project.id=8959cf36bec74b8ca22049270ee47cb2) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:47.603313 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01bc6d6-5a78-4bcb-915f-215ac4ef34a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:47.604921 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01bc6d6-5a78-4bcb-915f-215ac4ef34a9 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:47.611517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 214/427] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 19 10:17:47 2022] GET /identity/v3/role_assignments?group.id=4d2c82261800420081b2b0721af10b12&scope.project.id=8959cf36bec74b8ca22049270ee47cb2&role.id=c44d082d88894b888e94797acf7655b7 => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:47.659676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765cdff-23ee-4d97-9bbe-33a041e76096 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:47.710691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:47.723023 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1b9a5-d8d4-4c92-8c85-245765557155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:47.728125 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1b9a5-d8d4-4c92-8c85-245765557155 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:47.728125 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1b9a5-d8d4-4c92-8c85-245765557155 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:47.728125 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1b9a5-d8d4-4c92-8c85-245765557155 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:47.842951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1b9a5-d8d4-4c92-8c85-245765557155 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:47.844443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1b9a5-d8d4-4c92-8c85-245765557155 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:47.849842 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e4180-7529-40ae-adf4-2b1d7f5535cb None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:47.850448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e4180-7529-40ae-adf4-2b1d7f5535cb None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:47.850923 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e4180-7529-40ae-adf4-2b1d7f5535cb None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:47.852753 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 215/428] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:47.858366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace1d73-8203-4c13-92fe-3e23a5ecb962 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:47.858635 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace1d73-8203-4c13-92fe-3e23a5ecb962 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:47.859416 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace1d73-8203-4c13-92fe-3e23a5ecb962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:47.884160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-0ace1d73-8203-4c13-92fe-3e23a5ecb962 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:47.886822 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0ace1d73-8203-4c13-92fe-3e23a5ecb962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:47.977204 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 216/429] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:47.977812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 214/430] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 10:17:47.984042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace1d73-8203-4c13-92fe-3e23a5ecb962 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:48.041567 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:48.086798 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace1d73-8203-4c13-92fe-3e23a5ecb962 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:48.125050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909560fd-045b-4119-ae7d-33d252ff2411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:48.125702 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909560fd-045b-4119-ae7d-33d252ff2411 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:48.125978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909560fd-045b-4119-ae7d-33d252ff2411 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:48.126249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909560fd-045b-4119-ae7d-33d252ff2411 None admin] PATH_INFO: `/v3/projects/8959cf36bec74b8ca22049270ee47cb2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:48.127369 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909560fd-045b-4119-ae7d-33d252ff2411 None admin] RBAC: Authorizing `identity:get_project(project_id=8959cf36bec74b8ca22049270ee47cb2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:48.128803 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909560fd-045b-4119-ae7d-33d252ff2411 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:48.130728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 217/431] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:47 2022] GET /identity/v3/projects/8959cf36bec74b8ca22049270ee47cb2 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:48.136376 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1b9a5-d8d4-4c92-8c85-245765557155 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:48.168139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:48.254615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1b9a5-d8d4-4c92-8c85-245765557155 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:48.288103 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909560fd-045b-4119-ae7d-33d252ff2411 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:48.295750 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882e9d6-8508-431a-a130-755ecc957fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:48.297003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882e9d6-8508-431a-a130-755ecc957fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:48.297533 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882e9d6-8508-431a-a130-755ecc957fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:48.298003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882e9d6-8508-431a-a130-755ecc957fc5 None admin] PATH_INFO: `/v3/groups/4d2c82261800420081b2b0721af10b12` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:48.299342 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882e9d6-8508-431a-a130-755ecc957fc5 None admin] RBAC: Authorizing `identity:get_group(group_id=4d2c82261800420081b2b0721af10b12)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:48.300982 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882e9d6-8508-431a-a130-755ecc957fc5 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:48.303095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 215/432] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 19 10:17:48 2022] GET /identity/v3/groups/4d2c82261800420081b2b0721af10b12 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:48.310250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882e9d6-8508-431a-a130-755ecc957fc5 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:48.426448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909560fd-045b-4119-ae7d-33d252ff2411 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:48.430336 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:48.438326 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882e9d6-8508-431a-a130-755ecc957fc5 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:48.468643 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:48.478039 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604136b-caf9-4aa7-9f20-c32a1afcd7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:48.478914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604136b-caf9-4aa7-9f20-c32a1afcd7be None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:48.479158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604136b-caf9-4aa7-9f20-c32a1afcd7be None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:48.479435 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604136b-caf9-4aa7-9f20-c32a1afcd7be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:48.485447 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325dbecc-6e57-4879-8698-7aa8750dac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:48.486055 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dbecc-6e57-4879-8698-7aa8750dac4a None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:48.486491 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dbecc-6e57-4879-8698-7aa8750dac4a None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:48.486766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dbecc-6e57-4879-8698-7aa8750dac4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:48.487770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325dbecc-6e57-4879-8698-7aa8750dac4a None admin] RBAC: Adding query filter params (group.id=4d2c82261800420081b2b0721af10b12, role.id=e4309dcb11f74e10ba108b7b00264476, scope.project.id=8959cf36bec74b8ca22049270ee47cb2) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:48.488080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325dbecc-6e57-4879-8698-7aa8750dac4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:48.489561 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325dbecc-6e57-4879-8698-7aa8750dac4a None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:48.489790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604136b-caf9-4aa7-9f20-c32a1afcd7be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:48.494973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-9604136b-caf9-4aa7-9f20-c32a1afcd7be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:48.494973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:48.494973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:48.494973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:48.494973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:48.494973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:48.494973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:48.494973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:48.494973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:48.494973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:48.494973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:48.494973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:48.494973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:48.494973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:48.494973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:48.494973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:48.496086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:48.496086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:48.496086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:48.496086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:48.496086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:48.496086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:48.496086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:48.496086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:48.496086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:48.496086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:48.496086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:48.496086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:48.496086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:48.496086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:48.496086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:48.496086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:48.496879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:48.496879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:48.496879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:48.496879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:48.496879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:48.496879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:48.496879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:48.496879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:48.496879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:48.496879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:48.496879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:48.496879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:48.496879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 218/433] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:17:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:48.498501 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f273a01-975d-4398-a828-d3ec3f35f9bc None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:48.498501 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f273a01-975d-4398-a828-d3ec3f35f9bc None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:48.498632 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f273a01-975d-4398-a828-d3ec3f35f9bc None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:48.500036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 219/434] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:48.503546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 216/435] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 19 10:17:48 2022] GET /identity/v3/role_assignments?group.id=4d2c82261800420081b2b0721af10b12&scope.project.id=8959cf36bec74b8ca22049270ee47cb2&role.id=e4309dcb11f74e10ba108b7b00264476 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:48.506347 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2357d-b667-4ff3-a7b2-93484bdaed16 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:48.506347 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2357d-b667-4ff3-a7b2-93484bdaed16 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:48.506347 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2357d-b667-4ff3-a7b2-93484bdaed16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:48.512073 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325dbecc-6e57-4879-8698-7aa8750dac4a None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:48.532237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-73b2357d-b667-4ff3-a7b2-93484bdaed16 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:48.533962 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-73b2357d-b667-4ff3-a7b2-93484bdaed16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:48.629609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 220/436] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:17:48.639615 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df13da7-162b-4e64-869d-09d1f06c29dc None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:48.639615 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df13da7-162b-4e64-869d-09d1f06c29dc None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:48.639871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df13da7-162b-4e64-869d-09d1f06c29dc None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:48.641571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 221/437] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:17:48.647385 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a45ac-db13-4626-8a6e-ceaa784a414e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:48.647667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a45ac-db13-4626-8a6e-ceaa784a414e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:48.647953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a45ac-db13-4626-8a6e-ceaa784a414e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:48.655741 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325dbecc-6e57-4879-8698-7aa8750dac4a None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:48.668784 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7b8a45ac-db13-4626-8a6e-ceaa784a414e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:48.670723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7b8a45ac-db13-4626-8a6e-ceaa784a414e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:48.705419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dec63-030e-443e-af35-1cf26b7821ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:48.706038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dec63-030e-443e-af35-1cf26b7821ca None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:48.706320 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dec63-030e-443e-af35-1cf26b7821ca None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:48.706593 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dec63-030e-443e-af35-1cf26b7821ca None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:48.707261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3dec63-030e-443e-af35-1cf26b7821ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:48.707565 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3dec63-030e-443e-af35-1cf26b7821ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:48.708932 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3dec63-030e-443e-af35-1cf26b7821ca None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:48.721197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 217/438] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:17:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:48.726666 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dec63-030e-443e-af35-1cf26b7821ca None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:48.755637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:48.767578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 222/439] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:48.849440 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dec63-030e-443e-af35-1cf26b7821ca None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:48.885226 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e61b8-843c-400c-bad0-a1098dadd1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:48.885839 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e61b8-843c-400c-bad0-a1098dadd1be None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:48.886116 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e61b8-843c-400c-bad0-a1098dadd1be None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:48.886391 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e61b8-843c-400c-bad0-a1098dadd1be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:48.908212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a45ac-db13-4626-8a6e-ceaa784a414e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:48.970886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e61b8-843c-400c-bad0-a1098dadd1be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:48.972453 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e61b8-843c-400c-bad0-a1098dadd1be None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:48.978620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:48.998387 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:49.041728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a45ac-db13-4626-8a6e-ceaa784a414e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:49.076205 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c55c4-fbce-4985-87ab-6585e3798e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:49.076834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c55c4-fbce-4985-87ab-6585e3798e06 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:49.077149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c55c4-fbce-4985-87ab-6585e3798e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:49.077430 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c55c4-fbce-4985-87ab-6585e3798e06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:49.082649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 218/440] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 10:17:49.083486 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e61b8-843c-400c-bad0-a1098dadd1be None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:49.087839 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c55c4-fbce-4985-87ab-6585e3798e06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:49.093394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-295c55c4-fbce-4985-87ab-6585e3798e06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:49.093394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:49.093394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:49.093394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:49.093394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:49.093394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:49.093394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:49.093394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:49.093394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:49.093394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:49.093394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:49.093394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:49.093394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:49.093394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:49.093394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:49.093394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:49.094305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:49.094305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:49.094305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:49.094305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:49.094305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:49.094305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:49.094305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:49.094305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:49.094305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:49.094305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:49.094305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:49.094305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:49.094305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:49.094305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:49.094305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:49.094305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:49.095237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:49.095237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:49.095237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:49.095237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:49.095237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:49.095237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:49.095237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:49.095237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:49.095237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:49.095237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:49.095237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 10:17:49.095237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:49.095237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 223/441] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:17:49.095237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c55c4-fbce-4985-87ab-6585e3798e06 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:49.186209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e61b8-843c-400c-bad0-a1098dadd1be None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:49.193712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c55c4-fbce-4985-87ab-6585e3798e06 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:49.195989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:49.208700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:49.220465 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92784b69-5d74-4612-89cd-cb6e1e6d64db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:49.221115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92784b69-5d74-4612-89cd-cb6e1e6d64db None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:49.221393 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92784b69-5d74-4612-89cd-cb6e1e6d64db None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:49.221665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92784b69-5d74-4612-89cd-cb6e1e6d64db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:49.228746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4ed3b-9cb9-40e7-8b71-666671dbba32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:49.229384 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4ed3b-9cb9-40e7-8b71-666671dbba32 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:49.229662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4ed3b-9cb9-40e7-8b71-666671dbba32 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:49.229935 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4ed3b-9cb9-40e7-8b71-666671dbba32 None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:49.231102 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4ed3b-9cb9-40e7-8b71-666671dbba32 None admin] RBAC: Authorizing `identity:get_project(project_id=a52b1cb46c36451498d5787d735e4f12)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:49.231707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92784b69-5d74-4612-89cd-cb6e1e6d64db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:49.232576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4ed3b-9cb9-40e7-8b71-666671dbba32 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:49.234382 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 224/442] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:49 2022] GET /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:49.238844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4ed3b-9cb9-40e7-8b71-666671dbba32 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:49.238975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-92784b69-5d74-4612-89cd-cb6e1e6d64db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:17:49.238975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:49.238975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:49.238975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:49.238975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:49.238975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:49.238975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:49.238975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:49.238975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:49.238975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:49.238975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:49.238975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:49.238975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:49.238975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:49.238975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:49.238975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:49.240009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:49.240009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:49.240009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:49.240009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:49.240009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:49.240009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:49.240009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:49.240009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:49.240009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:49.240009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:49.240009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:49.240009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:49.240009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:49.240009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:49.240009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:49.240009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:49.240818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:49.240818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:49.240818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:49.240818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:49.240818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:49.240818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:49.240818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:49.240818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:49.240818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:49.240818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:49.240818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:17:49.240818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:49.240818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 219/443] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:17:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:49.240818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92784b69-5d74-4612-89cd-cb6e1e6d64db None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:49.334469 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4ed3b-9cb9-40e7-8b71-666671dbba32 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:49.338145 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92784b69-5d74-4612-89cd-cb6e1e6d64db None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:49.370521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe0cea-091f-4c6e-8f70-aae7a863a918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:49.371117 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe0cea-091f-4c6e-8f70-aae7a863a918 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:49.371399 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe0cea-091f-4c6e-8f70-aae7a863a918 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:49.371676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe0cea-091f-4c6e-8f70-aae7a863a918 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:49.372821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe0cea-091f-4c6e-8f70-aae7a863a918 None admin] RBAC: Adding query filter params (name=member) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:49.373152 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe0cea-091f-4c6e-8f70-aae7a863a918 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:49.373243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac305d2b-3473-4301-a4b5-fd694fd260bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:49.373872 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac305d2b-3473-4301-a4b5-fd694fd260bf None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:49.374152 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac305d2b-3473-4301-a4b5-fd694fd260bf None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:49.374425 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac305d2b-3473-4301-a4b5-fd694fd260bf None admin] PATH_INFO: `/v3/groups/4d2c82261800420081b2b0721af10b12` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:49.374545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe0cea-091f-4c6e-8f70-aae7a863a918 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:49.375537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac305d2b-3473-4301-a4b5-fd694fd260bf None admin] RBAC: Authorizing `identity:get_group(group_id=4d2c82261800420081b2b0721af10b12)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:49.377099 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac305d2b-3473-4301-a4b5-fd694fd260bf None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:49.378665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 220/444] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 19 10:17:49 2022] GET /identity/v3/groups/4d2c82261800420081b2b0721af10b12 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:49.379429 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac305d2b-3473-4301-a4b5-fd694fd260bf None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:49.387512 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 225/445] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:17:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:49.388365 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe0cea-091f-4c6e-8f70-aae7a863a918 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:49.426578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:49.430271 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:49.477950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac305d2b-3473-4301-a4b5-fd694fd260bf None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:49.491379 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe0cea-091f-4c6e-8f70-aae7a863a918 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:49.511048 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe9a24-cf6b-48b3-bfe2-fac23b160e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:49.511648 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe9a24-cf6b-48b3-bfe2-fac23b160e92 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:49.511926 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe9a24-cf6b-48b3-bfe2-fac23b160e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:49.512202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe9a24-cf6b-48b3-bfe2-fac23b160e92 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:49.514301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe9a24-cf6b-48b3-bfe2-fac23b160e92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:49.514301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe9a24-cf6b-48b3-bfe2-fac23b160e92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:49.514632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe9a24-cf6b-48b3-bfe2-fac23b160e92 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:49.527911 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 221/446] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:49.527911 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe9a24-cf6b-48b3-bfe2-fac23b160e92 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:49.528659 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7110b-4991-4ca2-b06c-232214a4a1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:49.528659 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7110b-4991-4ca2-b06c-232214a4a1ef None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:49.528659 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7110b-4991-4ca2-b06c-232214a4a1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:49.528659 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7110b-4991-4ca2-b06c-232214a4a1ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:49.528989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7110b-4991-4ca2-b06c-232214a4a1ef None admin] RBAC: Adding query filter params (group.id=4d2c82261800420081b2b0721af10b12, role.id=e4309dcb11f74e10ba108b7b00264476, scope.project.id=a52b1cb46c36451498d5787d735e4f12) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:49.528989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7110b-4991-4ca2-b06c-232214a4a1ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:49.530449 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7110b-4991-4ca2-b06c-232214a4a1ef None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:49.537374 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 226/447] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 19 10:17:49 2022] GET /identity/v3/role_assignments?group.id=4d2c82261800420081b2b0721af10b12&scope.project.id=a52b1cb46c36451498d5787d735e4f12&role.id=e4309dcb11f74e10ba108b7b00264476 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:49.538315 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7110b-4991-4ca2-b06c-232214a4a1ef None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:49.637930 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe9a24-cf6b-48b3-bfe2-fac23b160e92 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:49.647062 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:49.651847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7110b-4991-4ca2-b06c-232214a4a1ef None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:49.652404 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:49.677370 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9006d-e2f5-4950-a8da-0f4258361368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:49.678006 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9006d-e2f5-4950-a8da-0f4258361368 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:49.678383 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9006d-e2f5-4950-a8da-0f4258361368 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:49.678669 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9006d-e2f5-4950-a8da-0f4258361368 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:49.689206 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86859738-5eb3-4a28-8eff-11f9a1e9feeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:49.690130 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86859738-5eb3-4a28-8eff-11f9a1e9feeb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:49.690581 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86859738-5eb3-4a28-8eff-11f9a1e9feeb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:49.690997 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86859738-5eb3-4a28-8eff-11f9a1e9feeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:49.758273 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f9006d-e2f5-4950-a8da-0f4258361368 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:49.759995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f9006d-e2f5-4950-a8da-0f4258361368 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:49.771685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86859738-5eb3-4a28-8eff-11f9a1e9feeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:49.773268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86859738-5eb3-4a28-8eff-11f9a1e9feeb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:49.863889 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 222/448] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:49.869798 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9006d-e2f5-4950-a8da-0f4258361368 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:49.874401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:49.878794 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 227/449] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:17:49.884157 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86859738-5eb3-4a28-8eff-11f9a1e9feeb None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:49.887186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:49.971532 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9006d-e2f5-4950-a8da-0f4258361368 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:49.981916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86859738-5eb3-4a28-8eff-11f9a1e9feeb None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:50.004733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c614e7-6365-4e84-86b4-730ef592f853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:50.005393 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c614e7-6365-4e84-86b4-730ef592f853 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:50.005672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c614e7-6365-4e84-86b4-730ef592f853 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:50.005948 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c614e7-6365-4e84-86b4-730ef592f853 None admin] PATH_INFO: `/v3/projects/a52b1cb46c36451498d5787d735e4f12` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:50.007063 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c614e7-6365-4e84-86b4-730ef592f853 None admin] RBAC: Authorizing `identity:get_project(project_id=a52b1cb46c36451498d5787d735e4f12)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:50.008505 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c614e7-6365-4e84-86b4-730ef592f853 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:50.010314 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 223/450] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:49 2022] GET /identity/v3/projects/a52b1cb46c36451498d5787d735e4f12 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:50.014642 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bef434-aa92-498c-b2aa-3800aaf42a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:50.015348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bef434-aa92-498c-b2aa-3800aaf42a5f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:50.015634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bef434-aa92-498c-b2aa-3800aaf42a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:50.015908 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bef434-aa92-498c-b2aa-3800aaf42a5f None admin] PATH_INFO: `/v3/projects/1e8dac977aca46d48c891cd4c281d8cb` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:50.016130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c614e7-6365-4e84-86b4-730ef592f853 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:50.017019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bef434-aa92-498c-b2aa-3800aaf42a5f None admin] RBAC: Authorizing `identity:get_project(project_id=1e8dac977aca46d48c891cd4c281d8cb)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:50.018470 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bef434-aa92-498c-b2aa-3800aaf42a5f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:50.020061 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 228/451] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:17:49 2022] GET /identity/v3/projects/1e8dac977aca46d48c891cd4c281d8cb => generated 344 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:50.025561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bef434-aa92-498c-b2aa-3800aaf42a5f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:50.089635 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:50.096375 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:50.114671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c614e7-6365-4e84-86b4-730ef592f853 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:50.123780 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bef434-aa92-498c-b2aa-3800aaf42a5f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:50.148977 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c93bc-0ba2-4a7d-8b5c-9c6e4b2da4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:50.149880 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c93bc-0ba2-4a7d-8b5c-9c6e4b2da4de None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:50.150157 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c93bc-0ba2-4a7d-8b5c-9c6e4b2da4de None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:50.150429 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c93bc-0ba2-4a7d-8b5c-9c6e4b2da4de None admin] PATH_INFO: `/v3/groups/4d2c82261800420081b2b0721af10b12` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:50.151529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c93bc-0ba2-4a7d-8b5c-9c6e4b2da4de None admin] RBAC: Authorizing `identity:get_group(group_id=4d2c82261800420081b2b0721af10b12)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:50.152957 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c93bc-0ba2-4a7d-8b5c-9c6e4b2da4de None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:50.154716 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 224/452] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 19 10:17:50 2022] GET /identity/v3/groups/4d2c82261800420081b2b0721af10b12 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:50.157226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9365f5-a0ea-4395-abe4-7262aa10635b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:50.157845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9365f5-a0ea-4395-abe4-7262aa10635b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:50.158126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9365f5-a0ea-4395-abe4-7262aa10635b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:50.158518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9365f5-a0ea-4395-abe4-7262aa10635b None admin] PATH_INFO: `/v3/groups/944aa72aaa2649c8837445758b5f3b24` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:50.159638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9365f5-a0ea-4395-abe4-7262aa10635b None admin] RBAC: Authorizing `identity:get_group(group_id=944aa72aaa2649c8837445758b5f3b24)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:50.160532 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c93bc-0ba2-4a7d-8b5c-9c6e4b2da4de None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:50.161073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9365f5-a0ea-4395-abe4-7262aa10635b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:50.162630 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 229/453] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Sep 19 10:17:50 2022] GET /identity/v3/groups/944aa72aaa2649c8837445758b5f3b24 => generated 229 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:50.168053 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9365f5-a0ea-4395-abe4-7262aa10635b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:50.258346 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c93bc-0ba2-4a7d-8b5c-9c6e4b2da4de None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:50.267195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9365f5-a0ea-4395-abe4-7262aa10635b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:50.291491 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3865a-ce42-4968-b5db-9daf9d023e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:50.292092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3865a-ce42-4968-b5db-9daf9d023e3b None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:50.292371 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3865a-ce42-4968-b5db-9daf9d023e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:50.292644 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3865a-ce42-4968-b5db-9daf9d023e3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:50.293394 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3865a-ce42-4968-b5db-9daf9d023e3b None admin] RBAC: Adding query filter params (group.id=4d2c82261800420081b2b0721af10b12, role.id=c44d082d88894b888e94797acf7655b7, scope.project.id=a52b1cb46c36451498d5787d735e4f12) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:50.293699 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3865a-ce42-4968-b5db-9daf9d023e3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:50.295123 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3865a-ce42-4968-b5db-9daf9d023e3b None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:50.300314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0bf37-2256-4599-866b-7ca60307cdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:50.300648 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 225/454] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 19 10:17:50 2022] GET /identity/v3/role_assignments?group.id=4d2c82261800420081b2b0721af10b12&scope.project.id=a52b1cb46c36451498d5787d735e4f12&role.id=c44d082d88894b888e94797acf7655b7 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:50.303122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0bf37-2256-4599-866b-7ca60307cdc0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:50.303414 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0bf37-2256-4599-866b-7ca60307cdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:50.303693 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0bf37-2256-4599-866b-7ca60307cdc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:50.304449 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0bf37-2256-4599-866b-7ca60307cdc0 None admin] RBAC: Adding query filter params (group.id=944aa72aaa2649c8837445758b5f3b24, role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.project.id=1e8dac977aca46d48c891cd4c281d8cb) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:50.304791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0bf37-2256-4599-866b-7ca60307cdc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:50.306289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0bf37-2256-4599-866b-7ca60307cdc0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:50.308314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:50.313842 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 230/455] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Sep 19 10:17:50 2022] GET /identity/v3/role_assignments?group.id=944aa72aaa2649c8837445758b5f3b24&scope.project.id=1e8dac977aca46d48c891cd4c281d8cb&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:53.527229 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8eadb-006b-4a4d-8c2c-e365f84d5ca7 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:53.528811 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8eadb-006b-4a4d-8c2c-e365f84d5ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:53.529323 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8eadb-006b-4a4d-8c2c-e365f84d5ca7 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:53.531146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 226/456] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:53.536744 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0d256-9102-47f5-8037-3dbeefb56fbc None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:53.537613 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0d256-9102-47f5-8037-3dbeefb56fbc None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:53.538078 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0d256-9102-47f5-8037-3dbeefb56fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:53.562018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-bec0d256-9102-47f5-8037-3dbeefb56fbc None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:53.563518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bec0d256-9102-47f5-8037-3dbeefb56fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:53.567483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c32e8-d68c-4152-9e45-a5c83dc46bd7 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:53.568049 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c32e8-d68c-4152-9e45-a5c83dc46bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:53.568526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c32e8-d68c-4152-9e45-a5c83dc46bd7 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:53.570090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 227/457] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:53.582429 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801856b3-6fee-4420-acf2-f5618d830ae8 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:53.583033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801856b3-6fee-4420-acf2-f5618d830ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:53.583523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801856b3-6fee-4420-acf2-f5618d830ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:53.602993 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-801856b3-6fee-4420-acf2-f5618d830ae8 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:53.604513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-801856b3-6fee-4420-acf2-f5618d830ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:53.614247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:53.653614 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 231/458] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:53.655936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ac323-2fba-4d72-a392-80d936330f73 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:53.656369 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ac323-2fba-4d72-a392-80d936330f73 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:53.656800 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ac323-2fba-4d72-a392-80d936330f73 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:53.662394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 232/459] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:53.664152 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c81d11-a49e-4052-9795-526ae10fce60 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:53.664592 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c81d11-a49e-4052-9795-526ae10fce60 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:53.665117 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c81d11-a49e-4052-9795-526ae10fce60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:53.685699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-65c81d11-a49e-4052-9795-526ae10fce60 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:53.687103 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-65c81d11-a49e-4052-9795-526ae10fce60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:53.700914 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 228/460] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:53.788847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 233/461] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:17:54.070935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801856b3-6fee-4420-acf2-f5618d830ae8 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:54.123193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c81d11-a49e-4052-9795-526ae10fce60 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:54.195257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:54.208313 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801856b3-6fee-4420-acf2-f5618d830ae8 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:54.244458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c81d11-a49e-4052-9795-526ae10fce60 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:54.245552 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932d5db-bfdf-43a6-ab8a-7836443b708d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:54.246187 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932d5db-bfdf-43a6-ab8a-7836443b708d None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:54.246465 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932d5db-bfdf-43a6-ab8a-7836443b708d None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:54.247310 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932d5db-bfdf-43a6-ab8a-7836443b708d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:54.257756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932d5db-bfdf-43a6-ab8a-7836443b708d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:54.259238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932d5db-bfdf-43a6-ab8a-7836443b708d None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:54.261011 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:54.274665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-3932d5db-bfdf-43a6-ab8a-7836443b708d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:17:54.274665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:54.274665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:54.274665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:54.274665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:54.274665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:54.274665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:54.274665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:54.274665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:54.274665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:54.274665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:54.274665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:54.274665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:17:54.274665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:17:54.274665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:17:54.274665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:17:54.278207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:54.278207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:54.278207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:54.278207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:54.278207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:54.278207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:54.278207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:54.278207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:54.278207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:54.278207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:54.278207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:54.278207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:54.278207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:54.278207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:54.278207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:54.278207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:54.278207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:17:54.279080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:17:54.279080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:17:54.279080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:54.279080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 229/462] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:54.279080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932d5db-bfdf-43a6-ab8a-7836443b708d None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:54.287099 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaec9ba-2dae-4636-a225-1151855e3527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:54.287956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaec9ba-2dae-4636-a225-1151855e3527 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:54.288233 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaec9ba-2dae-4636-a225-1151855e3527 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:54.288529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaec9ba-2dae-4636-a225-1151855e3527 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:54.300898 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaec9ba-2dae-4636-a225-1151855e3527 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:54.302636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaec9ba-2dae-4636-a225-1151855e3527 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:54.319328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-dcaec9ba-2dae-4636-a225-1151855e3527 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:17:54.319328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:54.319328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:54.319328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:54.319328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:54.319328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:54.319328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:54.319328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:54.319328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:54.319328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:54.319328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:54.319328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:54.319328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:17:54.319328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:17:54.319328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:17:54.319328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:17:54.320257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:54.320257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:54.320257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:54.320257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:54.320257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:54.320257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:54.320257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:54.320257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:54.320257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:54.320257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:54.320257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:54.320257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:54.320257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:54.320257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:54.320257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:54.320257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:54.320257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:17:54.321189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:17:54.321189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:17:54.321189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:54.321189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 234/463] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:54.322118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f09a9f-f12f-4015-b0a4-b380ef13036f None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:54.322405 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f09a9f-f12f-4015-b0a4-b380ef13036f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:54.322679 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f09a9f-f12f-4015-b0a4-b380ef13036f None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:54.324128 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 235/464] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:54.325536 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f09a9f-f12f-4015-b0a4-b380ef13036f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:54.396896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932d5db-bfdf-43a6-ab8a-7836443b708d None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:54.431399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e67b7-f197-45d0-8e73-619e84c8818c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:54.432036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e67b7-f197-45d0-8e73-619e84c8818c None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:54.432318 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e67b7-f197-45d0-8e73-619e84c8818c None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:54.432592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e67b7-f197-45d0-8e73-619e84c8818c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:54.443252 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f09a9f-f12f-4015-b0a4-b380ef13036f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:54.443491 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0e67b7-f197-45d0-8e73-619e84c8818c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:54.444952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0e67b7-f197-45d0-8e73-619e84c8818c None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:54.455607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:54.460110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-6b0e67b7-f197-45d0-8e73-619e84c8818c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:17:54.460110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:54.460110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:54.460110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:54.460110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:54.460110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:54.460110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:54.460110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:54.460110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:54.460110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:54.460110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:54.460110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:54.460110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:17:54.460110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:17:54.460110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:17:54.460110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:17:54.461243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:54.461243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:54.461243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:54.461243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:54.461243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:54.461243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:54.461243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:54.461243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:54.461243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:54.461243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:54.461243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:54.461243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:54.461243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:54.461243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:54.461243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:54.461243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:54.461243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:17:54.462223 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:17:54.462223 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:17:54.462223 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:54.462223 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 230/465] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:54.467448 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e67b7-f197-45d0-8e73-619e84c8818c None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:54.482860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b66192-c880-423d-a57e-28b337c32912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:54.483467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b66192-c880-423d-a57e-28b337c32912 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:54.483748 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b66192-c880-423d-a57e-28b337c32912 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:54.484020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b66192-c880-423d-a57e-28b337c32912 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:54.484674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b66192-c880-423d-a57e-28b337c32912 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:54.484989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b66192-c880-423d-a57e-28b337c32912 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:54.486429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b66192-c880-423d-a57e-28b337c32912 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:54.503341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 236/466] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:54.503341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de27e74-b274-4f6c-82b5-e89e75b1e9e8 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:54.503341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de27e74-b274-4f6c-82b5-e89e75b1e9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:54.503341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de27e74-b274-4f6c-82b5-e89e75b1e9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:54.520668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-0de27e74-b274-4f6c-82b5-e89e75b1e9e8 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:54.521968 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0de27e74-b274-4f6c-82b5-e89e75b1e9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:54.523035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:54.578304 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e67b7-f197-45d0-8e73-619e84c8818c None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:54.596391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 237/467] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:54.597043 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de27e74-b274-4f6c-82b5-e89e75b1e9e8 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:54.619748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727bbc0-5696-4365-8ba6-b8458fb0ae60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:54.620725 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727bbc0-5696-4365-8ba6-b8458fb0ae60 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:54.621235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727bbc0-5696-4365-8ba6-b8458fb0ae60 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:54.622232 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727bbc0-5696-4365-8ba6-b8458fb0ae60 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:54.623638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727bbc0-5696-4365-8ba6-b8458fb0ae60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:54.624400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727bbc0-5696-4365-8ba6-b8458fb0ae60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:54.626837 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727bbc0-5696-4365-8ba6-b8458fb0ae60 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:54.652694 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 231/468] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:54.654310 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727bbc0-5696-4365-8ba6-b8458fb0ae60 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:54.710115 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:54.713310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de27e74-b274-4f6c-82b5-e89e75b1e9e8 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:54.748522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31309ce0-1306-4056-a850-eab28090d229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:54.749265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31309ce0-1306-4056-a850-eab28090d229 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:54.749648 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31309ce0-1306-4056-a850-eab28090d229 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:54.749998 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31309ce0-1306-4056-a850-eab28090d229 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:54.750722 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31309ce0-1306-4056-a850-eab28090d229 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:54.751095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31309ce0-1306-4056-a850-eab28090d229 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:54.752530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31309ce0-1306-4056-a850-eab28090d229 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:54.768525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 238/469] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:54.769740 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31309ce0-1306-4056-a850-eab28090d229 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:54.795842 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727bbc0-5696-4365-8ba6-b8458fb0ae60 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:54.805708 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:54.839594 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e62c7-5d79-4abf-912e-071f1ed86927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:54.840518 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e62c7-5d79-4abf-912e-071f1ed86927 None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:54.840963 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e62c7-5d79-4abf-912e-071f1ed86927 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:54.841551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e62c7-5d79-4abf-912e-071f1ed86927 None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:54.842199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e62c7-5d79-4abf-912e-071f1ed86927 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:54.843653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e62c7-5d79-4abf-912e-071f1ed86927 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:54.882256 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31309ce0-1306-4056-a850-eab28090d229 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:54.914843 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f5fe8-853c-4675-8121-7cf37577366c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:54.915434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f5fe8-853c-4675-8121-7cf37577366c None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:54.915739 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f5fe8-853c-4675-8121-7cf37577366c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:54.916005 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f5fe8-853c-4675-8121-7cf37577366c None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:54.916631 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f5fe8-853c-4675-8121-7cf37577366c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:54.918069 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f5fe8-853c-4675-8121-7cf37577366c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:54.949328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 232/470] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 19 10:17:54 2022] POST /identity/v3/users => generated 246 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:17:54.950757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e62c7-5d79-4abf-912e-071f1ed86927 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:54.997769 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 239/471] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 19 10:17:54 2022] POST /identity/v3/users => generated 248 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:17:55.000258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1bf7f-fce7-40f3-8cbe-5fe5be3e4f5b None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:55.000917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1bf7f-fce7-40f3-8cbe-5fe5be3e4f5b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:55.001487 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1bf7f-fce7-40f3-8cbe-5fe5be3e4f5b None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:55.003741 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 240/472] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:55.004967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1bf7f-fce7-40f3-8cbe-5fe5be3e4f5b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:55.037331 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:55.093036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e62c7-5d79-4abf-912e-071f1ed86927 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:55.148494 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31d4b1-26b2-4111-a49d-5fde938e3bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:55.150268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31d4b1-26b2-4111-a49d-5fde938e3bac None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:55.150268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31d4b1-26b2-4111-a49d-5fde938e3bac None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:55.150268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31d4b1-26b2-4111-a49d-5fde938e3bac None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:55.151013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31d4b1-26b2-4111-a49d-5fde938e3bac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:55.152530 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31d4b1-26b2-4111-a49d-5fde938e3bac None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:55.162724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1bf7f-fce7-40f3-8cbe-5fe5be3e4f5b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:55.207731 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d365b8-ee37-49fe-9c01-ee8c75e5a641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:55.208428 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d365b8-ee37-49fe-9c01-ee8c75e5a641 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:55.208711 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d365b8-ee37-49fe-9c01-ee8c75e5a641 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:55.208981 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d365b8-ee37-49fe-9c01-ee8c75e5a641 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:55.221081 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d365b8-ee37-49fe-9c01-ee8c75e5a641 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:55.222845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d365b8-ee37-49fe-9c01-ee8c75e5a641 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:55.238391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-84d365b8-ee37-49fe-9c01-ee8c75e5a641 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:17:55.238391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:55.238391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:55.238391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:55.238391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:55.238391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:55.238391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:55.238391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:55.238391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:55.238391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:55.238391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:55.238391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:55.238391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:17:55.238391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:17:55.238391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:17:55.238391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:17:55.239372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:55.239372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:55.239372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:55.239372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:55.239372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:55.239372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:55.239372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:55.239372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:55.239372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:55.239372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:55.239372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:55.239372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:55.239372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:55.239372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:55.239372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:55.239372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:55.239372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:17:55.240911 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:17:55.240911 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:17:55.240911 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:55.240911 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 241/473] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:17:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:55.241329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9b767-bf18-43ac-855b-1f56731c6967 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:55.241499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9b767-bf18-43ac-855b-1f56731c6967 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:55.241914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9b767-bf18-43ac-855b-1f56731c6967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:55.242026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 233/474] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 19 10:17:54 2022] POST /identity/v3/users => generated 248 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:17:55.245592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31d4b1-26b2-4111-a49d-5fde938e3bac None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:55.261633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5fc9b767-bf18-43ac-855b-1f56731c6967 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:55.263519 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5fc9b767-bf18-43ac-855b-1f56731c6967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:55.277536 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:55.368855 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:55.371283 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 242/475] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:55.396555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31d4b1-26b2-4111-a49d-5fde938e3bac None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:55.438470 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72140725-b95f-41ad-aa4d-124a9d6328c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:55.439105 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72140725-b95f-41ad-aa4d-124a9d6328c2 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:55.439392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72140725-b95f-41ad-aa4d-124a9d6328c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:55.439669 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72140725-b95f-41ad-aa4d-124a9d6328c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:55.440482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72140725-b95f-41ad-aa4d-124a9d6328c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:55.440788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72140725-b95f-41ad-aa4d-124a9d6328c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:55.442375 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72140725-b95f-41ad-aa4d-124a9d6328c2 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:55.458609 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 234/476] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:55.471071 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9b767-bf18-43ac-855b-1f56731c6967 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:55.592676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9b767-bf18-43ac-855b-1f56731c6967 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:55.637224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1592e7a-0c22-4940-99b5-d97f9100fa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:55.638303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1592e7a-0c22-4940-99b5-d97f9100fa21 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:55.638582 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1592e7a-0c22-4940-99b5-d97f9100fa21 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:55.638854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1592e7a-0c22-4940-99b5-d97f9100fa21 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:55.639505 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1592e7a-0c22-4940-99b5-d97f9100fa21 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:55.640943 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1592e7a-0c22-4940-99b5-d97f9100fa21 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:55.662708 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72140725-b95f-41ad-aa4d-124a9d6328c2 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:55.738132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 243/477] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 19 10:17:55 2022] POST /identity/v3/users => generated 249 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:17:55.757262 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:55.795711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72140725-b95f-41ad-aa4d-124a9d6328c2 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:55.846732 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a2855-b9f5-44bd-8f07-23260c96ad3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:55.847352 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a2855-b9f5-44bd-8f07-23260c96ad3b None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:55.847630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a2855-b9f5-44bd-8f07-23260c96ad3b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:55.847902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a2855-b9f5-44bd-8f07-23260c96ad3b None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:55.848600 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a2855-b9f5-44bd-8f07-23260c96ad3b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:55.850240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a2855-b9f5-44bd-8f07-23260c96ad3b None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:55.888629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.sql.core [None req-887a2855-b9f5-44bd-8f07-23260c96ad3b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 19 10:17:55.888629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 19 10:17:55.888629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [parameters: {'id': '2da84d4423fc475d9af14fd077653b8a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 19 10:17:55.888629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81563) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 19 10:17:55.906126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-887a2855-b9f5-44bd-8f07-23260c96ad3b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 19 10:17:55.906126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:55.906126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 10:17:55.906126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 10:17:55.906126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 10:17:55.906126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 10:17:55.906126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 10:17:55.906126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 10:17:55.906126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 10:17:55.906126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application conn.query(q) Sep 19 10:17:55.906126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 10:17:55.906126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 10:17:55.906126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 10:17:55.906126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application result.read() Sep 19 10:17:55.906126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 10:17:55.907089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 10:17:55.907089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 10:17:55.907089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 10:17:55.907089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 10:17:55.907089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 10:17:55.907089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 10:17:55.907089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 10:17:55.907089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 19 10:17:55.907089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:55.907089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 19 10:17:55.907089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:55.907089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:55.907089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 19 10:17:55.907089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 19 10:17:55.907089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 19 10:17:55.907089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return ref.to_dict() Sep 19 10:17:55.907089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 10:17:55.907089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application next(self.gen) Sep 19 10:17:55.908013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 19 10:17:55.908013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application yield resource Sep 19 10:17:55.908013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 10:17:55.908013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application next(self.gen) Sep 19 10:17:55.908013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 19 10:17:55.908013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self.session.rollback() Sep 19 10:17:55.908013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 10:17:55.908013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self.force_reraise() Sep 19 10:17:55.908013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 10:17:55.908013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise self.value Sep 19 10:17:55.908013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 19 10:17:55.908013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 19 10:17:55.908013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 19 10:17:55.908013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application session.commit() Sep 19 10:17:55.908013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 19 10:17:55.908013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 19 10:17:55.908013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 19 10:17:55.908867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._prepare_impl() Sep 19 10:17:55.908867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 19 10:17:55.908867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self.session.flush() Sep 19 10:17:55.908867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 19 10:17:55.908867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._flush(objects) Sep 19 10:17:55.908867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 19 10:17:55.908867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 19 10:17:55.908867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 19 10:17:55.908867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application compat.raise_( Sep 19 10:17:55.908867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 10:17:55.908867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception Sep 19 10:17:55.908867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 19 10:17:55.908867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application flush_context.execute() Sep 19 10:17:55.908867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 19 10:17:55.908867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rec.execute(self) Sep 19 10:17:55.908867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 19 10:17:55.908867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 19 10:17:55.910673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 19 10:17:55.910673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 19 10:17:55.910673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 19 10:17:55.910673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 19 10:17:55.910673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 19 10:17:55.910673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 19 10:17:55.910673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 19 10:17:55.910673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 19 10:17:55.910673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 19 10:17:55.910673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 19 10:17:55.910673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 19 10:17:55.910673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 19 10:17:55.910673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 19 10:17:55.910673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 19 10:17:55.910673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 10:17:55.910673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception Sep 19 10:17:55.912891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 10:17:55.912891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 10:17:55.912891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 10:17:55.912891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 10:17:55.912891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 10:17:55.912891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 10:17:55.912891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 10:17:55.912891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application conn.query(q) Sep 19 10:17:55.912891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 10:17:55.912891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 10:17:55.912891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 10:17:55.912891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application result.read() Sep 19 10:17:55.912891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 10:17:55.912891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 10:17:55.912891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 10:17:55.912891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 10:17:55.925901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 10:17:55.925901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 10:17:55.925901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 10:17:55.925901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 10:17:55.925901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 19 10:17:55.925901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 19 10:17:55.925901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [parameters: {'id': '2da84d4423fc475d9af14fd077653b8a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 19 10:17:55.925901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 19 10:17:55.925901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:55.925901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:17:55.925901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:55.925901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:55.925901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:55.925901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:55.925901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:55.925901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:55.927878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:55.927878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:55.927878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:55.927878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:55.927878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:55.927878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:55.927878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 19 10:17:55.927878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 19 10:17:55.927878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:55.927878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:55.927878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 19 10:17:55.927878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 19 10:17:55.927878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 19 10:17:55.927878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 19 10:17:55.927878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 19 10:17:55.927878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:55.929884 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 235/478] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Sep 19 10:17:55 2022] POST /identity/v3/roles => generated 144 bytes in 245 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:55.930019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1592e7a-0c22-4940-99b5-d97f9100fa21 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:56.013977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:56.060698 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1592e7a-0c22-4940-99b5-d97f9100fa21 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:56.096155 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d57f09-1962-46b5-806d-1379baae46bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:56.097249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d57f09-1962-46b5-806d-1379baae46bd None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:56.097745 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d57f09-1962-46b5-806d-1379baae46bd None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:56.098254 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d57f09-1962-46b5-806d-1379baae46bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:56.109846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d57f09-1962-46b5-806d-1379baae46bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:56.115458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-c4d57f09-1962-46b5-806d-1379baae46bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:56.115458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:56.115458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:56.115458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:56.115458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:56.115458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:56.115458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:56.115458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:56.115458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:56.115458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:56.115458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:56.115458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:56.115458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:56.115458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:56.115458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:56.115458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:56.116392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:56.116392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:56.116392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:56.116392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:56.116392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:56.116392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:56.116392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:56.116392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:56.116392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:56.116392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:56.116392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:56.116392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:56.116392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:56.116392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:56.116392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:56.116392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:56.117235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:56.117235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:56.117235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:56.117235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:56.117235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:56.117235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:56.117235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:56.117235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:56.117235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:56.117235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:56.117235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 10:17:56.117235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:56.117850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 244/479] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:17:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:56.123866 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a2855-b9f5-44bd-8f07-23260c96ad3b None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:56.229492 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a2855-b9f5-44bd-8f07-23260c96ad3b None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:56.267118 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50590a93-b961-4095-995f-7a7922a45f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:56.268074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50590a93-b961-4095-995f-7a7922a45f10 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:56.268549 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50590a93-b961-4095-995f-7a7922a45f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:56.269045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50590a93-b961-4095-995f-7a7922a45f10 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:56.269942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50590a93-b961-4095-995f-7a7922a45f10 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:56.270455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50590a93-b961-4095-995f-7a7922a45f10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:56.272067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50590a93-b961-4095-995f-7a7922a45f10 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:56.281272 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 236/480] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:17:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:57.588513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d4ba8-7de1-492f-8bea-24ff51f8fbee None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:57.589033 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d4ba8-7de1-492f-8bea-24ff51f8fbee None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:57.589033 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d4ba8-7de1-492f-8bea-24ff51f8fbee None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:57.590712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 245/481] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:57.596363 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef025c-d273-4bee-979e-b65284e9015b None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:57.596911 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef025c-d273-4bee-979e-b65284e9015b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:57.596911 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef025c-d273-4bee-979e-b65284e9015b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:57.604847 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:57.620846 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-a9ef025c-d273-4bee-979e-b65284e9015b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:57.622637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-a9ef025c-d273-4bee-979e-b65284e9015b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:57.712551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 237/482] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:57.749672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd212914-f8e0-4d32-b24d-27e7ff15ad2d None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:57.749977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd212914-f8e0-4d32-b24d-27e7ff15ad2d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:57.750230 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd212914-f8e0-4d32-b24d-27e7ff15ad2d None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:57.751559 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 246/483] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:57.756768 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3a44e-576d-43e8-95c2-76cb1a7eaab9 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:57.757154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3a44e-576d-43e8-95c2-76cb1a7eaab9 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:57.757432 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3a44e-576d-43e8-95c2-76cb1a7eaab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:57.776577 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-b7b3a44e-576d-43e8-95c2-76cb1a7eaab9 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:57.777860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b7b3a44e-576d-43e8-95c2-76cb1a7eaab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:57.871048 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 238/484] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:58.040388 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a92459-9a80-4bb4-80d4-8f5dfedfceb3 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:58.040771 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a92459-9a80-4bb4-80d4-8f5dfedfceb3 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:58.040950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a92459-9a80-4bb4-80d4-8f5dfedfceb3 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:58.042606 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 247/485] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:58.059493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dea17a-1d4f-4db0-b91f-ce8c8a1e2cf6 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:58.059778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dea17a-1d4f-4db0-b91f-ce8c8a1e2cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:58.060049 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dea17a-1d4f-4db0-b91f-ce8c8a1e2cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:58.068068 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a92459-9a80-4bb4-80d4-8f5dfedfceb3 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:58.088787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-b7dea17a-1d4f-4db0-b91f-ce8c8a1e2cf6 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:58.092205 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b7dea17a-1d4f-4db0-b91f-ce8c8a1e2cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:58.198152 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:58.214532 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 239/486] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:58.240166 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a92459-9a80-4bb4-80d4-8f5dfedfceb3 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:58.255042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:58.276270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f132e-310a-49f0-a2eb-5b47aa90deec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:58.277208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f132e-310a-49f0-a2eb-5b47aa90deec None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:58.278241 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f132e-310a-49f0-a2eb-5b47aa90deec None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:58.278692 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f132e-310a-49f0-a2eb-5b47aa90deec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:58.280331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dea17a-1d4f-4db0-b91f-ce8c8a1e2cf6 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:58.301291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f132e-310a-49f0-a2eb-5b47aa90deec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:58.306782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-675f132e-310a-49f0-a2eb-5b47aa90deec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:17:58.306782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:58.306782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:58.306782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:58.306782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:58.306782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:58.306782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:58.306782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:58.306782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:58.306782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:58.306782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:58.306782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:58.306782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:58.306782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:58.306782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:58.306782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:58.307667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:58.307667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:58.307667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:58.307667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:58.307667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:58.307667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:58.307667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:58.307667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:58.307667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:58.307667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:58.307667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:58.307667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:58.307667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:58.307667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:58.307667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:58.307667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:58.308503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:58.308503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:58.308503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:58.308503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:58.308503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:58.308503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:58.308503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:58.308503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:58.308503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:58.308503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:58.308503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:17:58.308503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:58.311023 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 248/487] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:17:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:17:58.321304 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f132e-310a-49f0-a2eb-5b47aa90deec None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:58.412551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dea17a-1d4f-4db0-b91f-ce8c8a1e2cf6 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:58.467933 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6727083-625d-4875-b8a3-aa58c3d4fcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:58.473165 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6727083-625d-4875-b8a3-aa58c3d4fcb1 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:58.473693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6727083-625d-4875-b8a3-aa58c3d4fcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:58.474171 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6727083-625d-4875-b8a3-aa58c3d4fcb1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:58.493566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f132e-310a-49f0-a2eb-5b47aa90deec None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:58.495961 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6727083-625d-4875-b8a3-aa58c3d4fcb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:58.500923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-a6727083-625d-4875-b8a3-aa58c3d4fcb1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:17:58.500923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:58.500923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:58.500923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:58.500923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:58.500923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:58.500923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:58.500923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:58.500923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:58.500923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:58.500923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:58.500923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:58.500923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:58.500923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:58.500923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:58.500923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:58.501874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:58.501874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:58.501874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:58.501874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:58.501874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:58.501874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:58.501874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:58.501874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:58.501874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:58.501874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:58.501874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:58.501874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:58.501874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:58.501874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:58.501874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:58.501874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:58.502756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:58.502756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:58.502756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:58.502756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:58.502756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:58.502756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:58.502756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:58.502756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:58.502756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:58.502756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:58.502756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:17:58.502756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:58.511561 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 240/488] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:17:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:17:58.517243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6727083-625d-4875-b8a3-aa58c3d4fcb1 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:58.536097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81e786-6494-43b5-a6ce-68ebe7710e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:58.536703 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81e786-6494-43b5-a6ce-68ebe7710e3c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:58.537051 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81e786-6494-43b5-a6ce-68ebe7710e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:58.537367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81e786-6494-43b5-a6ce-68ebe7710e3c None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:58.538040 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81e786-6494-43b5-a6ce-68ebe7710e3c None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:58.538335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81e786-6494-43b5-a6ce-68ebe7710e3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:58.539833 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81e786-6494-43b5-a6ce-68ebe7710e3c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:58.550158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 249/489] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:17:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:58.556281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81e786-6494-43b5-a6ce-68ebe7710e3c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:58.594643 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:58.650615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:58.674132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81e786-6494-43b5-a6ce-68ebe7710e3c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:58.713007 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c458f-16c8-496e-aaa1-4abbd27ecef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:58.713427 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6727083-625d-4875-b8a3-aa58c3d4fcb1 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:58.714186 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c458f-16c8-496e-aaa1-4abbd27ecef7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:58.714632 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c458f-16c8-496e-aaa1-4abbd27ecef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:58.715107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c458f-16c8-496e-aaa1-4abbd27ecef7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:58.727725 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c458f-16c8-496e-aaa1-4abbd27ecef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:58.729620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c458f-16c8-496e-aaa1-4abbd27ecef7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:58.758153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-dd2c458f-16c8-496e-aaa1-4abbd27ecef7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:17:58.758153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:58.758153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:58.758153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:58.758153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:58.758153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:58.758153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:58.758153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:58.758153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:58.758153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:58.758153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:58.758153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:58.758153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:17:58.758153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:17:58.758153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:17:58.758153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:17:58.759135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:58.759135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:58.759135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:58.759135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:58.759135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:58.759135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:58.759135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:58.759135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:58.759135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:58.759135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:58.759135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:58.759135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:58.759135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:58.759135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:58.759135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:58.759135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:58.759135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:17:58.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:17:58.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:17:58.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:58.760159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78debcec-cfb4-4abb-91fc-eff6a180006f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:58.760159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78debcec-cfb4-4abb-91fc-eff6a180006f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:58.760159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78debcec-cfb4-4abb-91fc-eff6a180006f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:58.760159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78debcec-cfb4-4abb-91fc-eff6a180006f None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:58.760430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78debcec-cfb4-4abb-91fc-eff6a180006f None admin] RBAC: Adding query filter params (name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:58.760497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78debcec-cfb4-4abb-91fc-eff6a180006f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:58.761948 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78debcec-cfb4-4abb-91fc-eff6a180006f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:58.762985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 250/490] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:58.768335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c458f-16c8-496e-aaa1-4abbd27ecef7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:58.773002 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 241/491] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:17:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:17:58.773370 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78debcec-cfb4-4abb-91fc-eff6a180006f None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:58.870228 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:58.882873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c458f-16c8-496e-aaa1-4abbd27ecef7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:58.890913 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78debcec-cfb4-4abb-91fc-eff6a180006f None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:58.911338 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:58.917836 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77ff63-4b3c-4b42-9f1d-40392dbf6d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:58.919431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77ff63-4b3c-4b42-9f1d-40392dbf6d2a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:58.919431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77ff63-4b3c-4b42-9f1d-40392dbf6d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:58.919719 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77ff63-4b3c-4b42-9f1d-40392dbf6d2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:58.930020 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03abf2-3869-478d-b280-2bd94c0b09b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:58.930624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03abf2-3869-478d-b280-2bd94c0b09b1 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:58.930904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03abf2-3869-478d-b280-2bd94c0b09b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:58.931249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03abf2-3869-478d-b280-2bd94c0b09b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:58.931672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77ff63-4b3c-4b42-9f1d-40392dbf6d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:58.931902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03abf2-3869-478d-b280-2bd94c0b09b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:58.932414 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03abf2-3869-478d-b280-2bd94c0b09b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:58.934885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03abf2-3869-478d-b280-2bd94c0b09b1 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:58.937936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-fd77ff63-4b3c-4b42-9f1d-40392dbf6d2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:17:58.937936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:58.937936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:58.937936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:58.937936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:58.937936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:58.937936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:58.937936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:58.937936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:58.937936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:58.937936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:58.937936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:58.937936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:58.937936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:58.937936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:58.937936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:58.939016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:58.939016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:58.939016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:58.939016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:58.939016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:58.939016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:58.939016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:58.939016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:58.939016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:58.939016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:58.939016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:58.939016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:58.939016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:58.939016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:58.939016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:58.939016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:58.940038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:58.940038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:58.940038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:58.940038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:58.940038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:58.940038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:58.940038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:58.940038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:58.940038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:58.940038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:58.940038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:17:58.940038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:58.941999 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 251/492] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:17:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:58.942904 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77ff63-4b3c-4b42-9f1d-40392dbf6d2a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:58.951709 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 242/493] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:58.952931 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88db61c-9b2f-4985-ae30-ad1d56519db1 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:58.953240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88db61c-9b2f-4985-ae30-ad1d56519db1 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:58.953513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88db61c-9b2f-4985-ae30-ad1d56519db1 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:58.958407 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 243/494] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:58.958407 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88db61c-9b2f-4985-ae30-ad1d56519db1 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:59.064463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77ff63-4b3c-4b42-9f1d-40392dbf6d2a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:59.065366 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88db61c-9b2f-4985-ae30-ad1d56519db1 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:59.102536 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472f3eb-5ac0-4ddd-b672-0fa6ee86fcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:59.103256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472f3eb-5ac0-4ddd-b672-0fa6ee86fcc3 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:59.103536 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472f3eb-5ac0-4ddd-b672-0fa6ee86fcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:59.103811 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472f3eb-5ac0-4ddd-b672-0fa6ee86fcc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:59.104477 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472f3eb-5ac0-4ddd-b672-0fa6ee86fcc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:59.104783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472f3eb-5ac0-4ddd-b672-0fa6ee86fcc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:59.106786 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472f3eb-5ac0-4ddd-b672-0fa6ee86fcc3 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:59.108155 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4b82d-80de-47ab-a1c2-519dedbef02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:59.109162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4b82d-80de-47ab-a1c2-519dedbef02e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:59.109665 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4b82d-80de-47ab-a1c2-519dedbef02e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:59.110111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4b82d-80de-47ab-a1c2-519dedbef02e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:59.114894 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 244/495] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:17:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:59.115706 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472f3eb-5ac0-4ddd-b672-0fa6ee86fcc3 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:59.123338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4b82d-80de-47ab-a1c2-519dedbef02e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:59.124779 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4b82d-80de-47ab-a1c2-519dedbef02e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:59.133282 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:59.139737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-75a4b82d-80de-47ab-a1c2-519dedbef02e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:17:59.139737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:59.139737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:59.139737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:59.139737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:59.139737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:59.139737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:59.139737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:59.139737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:59.139737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:59.139737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:59.139737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:59.139737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:17:59.139737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:17:59.139737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:17:59.139737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:17:59.143984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:59.143984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:59.143984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:59.143984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:59.143984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:59.143984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:59.143984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:59.143984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:59.143984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:59.143984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:59.143984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:59.143984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:59.143984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:59.143984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:59.143984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:59.143984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:59.143984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:17:59.144851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:17:59.144851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:17:59.144851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:59.144851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 252/496] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:59.144851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6cc07-14e1-4dbd-8fc0-ba4141952e51 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:59.144851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6cc07-14e1-4dbd-8fc0-ba4141952e51 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:59.144851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6cc07-14e1-4dbd-8fc0-ba4141952e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:59.160933 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-feb6cc07-14e1-4dbd-8fc0-ba4141952e51 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:59.161942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:59.162575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-feb6cc07-14e1-4dbd-8fc0-ba4141952e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:59.220431 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472f3eb-5ac0-4ddd-b672-0fa6ee86fcc3 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:59.227056 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 253/497] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:59.228289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1878e2-7843-4302-84f7-42def44e28ba None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:59.228577 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1878e2-7843-4302-84f7-42def44e28ba None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:59.228854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1878e2-7843-4302-84f7-42def44e28ba None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:59.230266 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 254/498] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:17:59.231388 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1878e2-7843-4302-84f7-42def44e28ba None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:59.255768 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3dab4-8e4d-4fa9-9dd3-f1440c5bd5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:59.256728 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3dab4-8e4d-4fa9-9dd3-f1440c5bd5cf None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:59.257242 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3dab4-8e4d-4fa9-9dd3-f1440c5bd5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:59.257723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3dab4-8e4d-4fa9-9dd3-f1440c5bd5cf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:59.267104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3dab4-8e4d-4fa9-9dd3-f1440c5bd5cf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:59.269024 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3dab4-8e4d-4fa9-9dd3-f1440c5bd5cf None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:59.283213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-bbf3dab4-8e4d-4fa9-9dd3-f1440c5bd5cf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 10:17:59.283213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:59.283213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:59.283213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:59.283213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:59.283213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:59.283213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:59.283213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:59.283213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:59.283213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:59.283213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:59.283213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:59.283213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:17:59.283213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:17:59.283213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:17:59.283213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:17:59.284111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:59.284111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:59.284111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:17:59.284111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:17:59.284111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:17:59.284111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:17:59.284111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:59.284111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:59.284111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:59.284111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:59.284111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:59.284111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:59.284111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:59.284111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:59.284111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:59.284111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:59.284111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:59.284111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:59.285005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:59.285005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:59.285005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:17:59.285005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:17:59.285005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:17:59.285005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:17:59.285005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:17:59.285005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:17:59.285005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 10:17:59.285005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:59.289199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 245/499] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Sep 19 10:17:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:59.290090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3dab4-8e4d-4fa9-9dd3-f1440c5bd5cf None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:59.348073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1878e2-7843-4302-84f7-42def44e28ba None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:59.377197 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:59.382862 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de6121-1e3d-41ca-b18f-472a271c9393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:59.383480 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de6121-1e3d-41ca-b18f-472a271c9393 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:59.383756 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de6121-1e3d-41ca-b18f-472a271c9393 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:59.384171 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de6121-1e3d-41ca-b18f-472a271c9393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:59.394167 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de6121-1e3d-41ca-b18f-472a271c9393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:59.395695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de6121-1e3d-41ca-b18f-472a271c9393 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:59.400098 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:59.405487 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3dab4-8e4d-4fa9-9dd3-f1440c5bd5cf None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:59.414451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-33de6121-1e3d-41ca-b18f-472a271c9393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:17:59.414451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:59.414451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:59.414451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:59.414451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:59.414451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:59.414451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:59.414451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:59.414451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:59.414451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:59.414451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:59.414451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:59.414451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:17:59.414451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:17:59.414451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:17:59.414451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:17:59.415602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:59.415602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:59.415602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:59.415602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:59.415602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:59.415602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:59.415602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:59.415602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:59.415602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:59.415602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:59.415602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:59.415602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:59.415602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:59.415602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:59.415602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:59.415602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:59.415602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:17:59.416455 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:17:59.416455 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:17:59.416455 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:59.420616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 255/500] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:17:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:59.420616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20433477-324c-400e-b459-fbb9d9227429 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:59.420616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20433477-324c-400e-b459-fbb9d9227429 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:59.420616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20433477-324c-400e-b459-fbb9d9227429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:59.439502 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-20433477-324c-400e-b459-fbb9d9227429 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:17:59.441568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-20433477-324c-400e-b459-fbb9d9227429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:17:59.445930 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fa64d-a0f3-4e9f-8c62-3dcbba392c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:59.446531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fa64d-a0f3-4e9f-8c62-3dcbba392c69 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:59.446803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fa64d-a0f3-4e9f-8c62-3dcbba392c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:59.447070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fa64d-a0f3-4e9f-8c62-3dcbba392c69 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:59.447697 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fa64d-a0f3-4e9f-8c62-3dcbba392c69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:59.447993 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fa64d-a0f3-4e9f-8c62-3dcbba392c69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:59.449914 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fa64d-a0f3-4e9f-8c62-3dcbba392c69 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:59.466234 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 246/501] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:59.466234 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fa64d-a0f3-4e9f-8c62-3dcbba392c69 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:59.513854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 256/502] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:17:59.514709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20433477-324c-400e-b459-fbb9d9227429 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:59.570965 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fa64d-a0f3-4e9f-8c62-3dcbba392c69 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:59.608148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d5aa1-3036-45b3-93e4-9fa53caaf4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:59.608679 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d5aa1-3036-45b3-93e4-9fa53caaf4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:59.608959 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d5aa1-3036-45b3-93e4-9fa53caaf4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:59.609261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d5aa1-3036-45b3-93e4-9fa53caaf4a4 None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:59.609985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d5aa1-3036-45b3-93e4-9fa53caaf4a4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:59.610289 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d5aa1-3036-45b3-93e4-9fa53caaf4a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:59.611714 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d5aa1-3036-45b3-93e4-9fa53caaf4a4 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:59.618266 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20433477-324c-400e-b459-fbb9d9227429 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:59.631247 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:59.652332 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 247/503] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Sep 19 10:17:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:59.653365 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1f096-b785-4ee5-b2c9-34a7d15268eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:59.653787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d5aa1-3036-45b3-93e4-9fa53caaf4a4 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:59.653887 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1f096-b785-4ee5-b2c9-34a7d15268eb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:59.654081 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1f096-b785-4ee5-b2c9-34a7d15268eb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:59.654327 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1f096-b785-4ee5-b2c9-34a7d15268eb None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:59.654968 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1f096-b785-4ee5-b2c9-34a7d15268eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:17:59.655325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1f096-b785-4ee5-b2c9-34a7d15268eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:59.656693 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1f096-b785-4ee5-b2c9-34a7d15268eb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:59.673014 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 257/504] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:17:59.673014 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1f096-b785-4ee5-b2c9-34a7d15268eb None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:59.673962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:59.757921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d5aa1-3036-45b3-93e4-9fa53caaf4a4 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:59.779667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1f096-b785-4ee5-b2c9-34a7d15268eb None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:59.793852 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bf172-9345-446e-8f30-a771b7d6e036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:59.794476 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bf172-9345-446e-8f30-a771b7d6e036 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:59.794752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bf172-9345-446e-8f30-a771b7d6e036 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:59.795024 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bf172-9345-446e-8f30-a771b7d6e036 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:59.802061 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55bf172-9345-446e-8f30-a771b7d6e036 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:59.803524 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55bf172-9345-446e-8f30-a771b7d6e036 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:59.815966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-b55bf172-9345-446e-8f30-a771b7d6e036 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 10:17:59.815966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:59.815966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:59.815966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:59.815966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:59.815966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:59.815966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:59.815966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:59.815966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:59.815966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:59.815966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:59.815966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:59.815966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:17:59.815966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:17:59.815966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:17:59.815966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:17:59.817039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:59.817039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:59.817039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:17:59.817039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:17:59.817039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:17:59.817039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:17:59.817039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:59.817039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:59.817039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:59.817039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:59.817039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:59.817039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:59.817039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:59.817039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:59.817039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:59.817039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:59.817039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:59.817039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:59.817976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:59.817976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:59.817976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:17:59.817976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:17:59.817976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:17:59.817976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:17:59.817976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:17:59.817976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:17:59.817976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 10:17:59.817976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:17:59.818490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b3d1c-6821-49dd-bd9b-225f30a5de84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:59.818490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b3d1c-6821-49dd-bd9b-225f30a5de84 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:59.818490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b3d1c-6821-49dd-bd9b-225f30a5de84 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:59.818697 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 248/505] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:59.818763 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b3d1c-6821-49dd-bd9b-225f30a5de84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:59.818828 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bf172-9345-446e-8f30-a771b7d6e036 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:59.832618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b3d1c-6821-49dd-bd9b-225f30a5de84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:17:59.837992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-8a5b3d1c-6821-49dd-bd9b-225f30a5de84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:17:59.837992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:59.837992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:59.837992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:59.837992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:59.837992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:59.837992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:59.837992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:59.837992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:59.837992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:59.837992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:59.837992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:59.837992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:17:59.837992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:17:59.837992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:17:59.837992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:17:59.839125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:17:59.839125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:17:59.839125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:17:59.839125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:17:59.839125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:59.839125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:59.839125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:59.839125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:59.839125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:59.839125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:59.839125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:59.839125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:59.839125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:59.839125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:59.839125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:59.839125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:59.841562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:59.841562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:59.841562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:59.841562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:59.841562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:17:59.841562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:17:59.841562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:17:59.841562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:17:59.841562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:17:59.841562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:17:59.841562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:17:59.841562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:59.841562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 258/506] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:17:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:59.841562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b3d1c-6821-49dd-bd9b-225f30a5de84 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:17:59.870910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:59.895031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:17:59.916783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bf172-9345-446e-8f30-a771b7d6e036 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:59.938541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b3d1c-6821-49dd-bd9b-225f30a5de84 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:17:59.950483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df19a6-ef90-4f2d-a55c-beabd4a80c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:59.951086 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df19a6-ef90-4f2d-a55c-beabd4a80c6c None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:59.951367 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df19a6-ef90-4f2d-a55c-beabd4a80c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:59.951643 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df19a6-ef90-4f2d-a55c-beabd4a80c6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:59.972139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ea8d0-a8eb-4bb1-af95-5b41b6354169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:17:59.972827 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ea8d0-a8eb-4bb1-af95-5b41b6354169 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:17:59.973137 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ea8d0-a8eb-4bb1-af95-5b41b6354169 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:17:59.973412 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ea8d0-a8eb-4bb1-af95-5b41b6354169 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:17:59.980131 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ea8d0-a8eb-4bb1-af95-5b41b6354169 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:17:59.981605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ea8d0-a8eb-4bb1-af95-5b41b6354169 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:17:59.994008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-467ea8d0-a8eb-4bb1-af95-5b41b6354169 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 19 10:17:59.994008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:17:59.994008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:17:59.994008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:17:59.994008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:17:59.994008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:17:59.994008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:17:59.994008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:17:59.994008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:17:59.994008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:17:59.994008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:17:59.994008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:17:59.994008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:17:59.994008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:17:59.994008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:17:59.994008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:17:59.995046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:17:59.995046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:17:59.995046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:17:59.995046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:17:59.995046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:17:59.995046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:17:59.995046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:17:59.995046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:17:59.995046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:17:59.995046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:17:59.995046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:17:59.995046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:17:59.995046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:17:59.995046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:17:59.995046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:17:59.995046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:17:59.995046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:17:59.995046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:17:59.995957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:17:59.995957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:17:59.995957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:17:59.995957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:17:59.995957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:17:59.995957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:17:59.995957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:17:59.995957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:17:59.995957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 19 10:17:59.995957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:17:59.995957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 259/507] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:17:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:17:59.995957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ea8d0-a8eb-4bb1-af95-5b41b6354169 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:00.018001 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df19a6-ef90-4f2d-a55c-beabd4a80c6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:00.019451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df19a6-ef90-4f2d-a55c-beabd4a80c6c None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:00.093940 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:00.097350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:00.103208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ea8d0-a8eb-4bb1-af95-5b41b6354169 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:00.119698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 249/508] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:17:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:18:00.120524 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df19a6-ef90-4f2d-a55c-beabd4a80c6c None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:00.138326 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce63f7-5f40-4333-ab12-fd9801297fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:00.138873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce63f7-5f40-4333-ab12-fd9801297fcb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:00.139157 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce63f7-5f40-4333-ab12-fd9801297fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:00.139432 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce63f7-5f40-4333-ab12-fd9801297fcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:00.149674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce63f7-5f40-4333-ab12-fd9801297fcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:00.151140 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce63f7-5f40-4333-ab12-fd9801297fcb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:00.165236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-c8ce63f7-5f40-4333-ab12-fd9801297fcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:00.165236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:00.165236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:00.165236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:00.165236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:00.165236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:00.165236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:00.165236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:00.165236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:00.165236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:00.165236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:00.165236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:00.165236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:00.165236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:00.165236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:00.165236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:00.166360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:00.166360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:00.166360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:00.166360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:00.166360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:00.166360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:00.166360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:00.166360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:00.166360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:00.166360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:00.166360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:00.166360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:00.166360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:00.166360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:00.166360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:00.166360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:00.166360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:00.167449 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:00.167449 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:00.167449 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:00.167449 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 260/509] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:17:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:00.167449 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce63f7-5f40-4333-ab12-fd9801297fcb None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:00.219947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df19a6-ef90-4f2d-a55c-beabd4a80c6c None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:00.253547 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2b203-65ee-461c-b9f9-569e308a78b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:00.254148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2b203-65ee-461c-b9f9-569e308a78b6 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:00.254426 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2b203-65ee-461c-b9f9-569e308a78b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:00.254709 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2b203-65ee-461c-b9f9-569e308a78b6 None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:00.255427 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2b203-65ee-461c-b9f9-569e308a78b6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:00.255735 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2b203-65ee-461c-b9f9-569e308a78b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:00.257189 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2b203-65ee-461c-b9f9-569e308a78b6 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:00.262650 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce63f7-5f40-4333-ab12-fd9801297fcb None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:00.274235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 250/510] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Sep 19 10:18:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:00.275025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2b203-65ee-461c-b9f9-569e308a78b6 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:00.300076 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1bd71-9152-4571-b2d9-199101e9d540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:00.300668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1bd71-9152-4571-b2d9-199101e9d540 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:00.300977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1bd71-9152-4571-b2d9-199101e9d540 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:00.301273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1bd71-9152-4571-b2d9-199101e9d540 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:00.301917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1bd71-9152-4571-b2d9-199101e9d540 None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:00.302217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1bd71-9152-4571-b2d9-199101e9d540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:00.303559 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1bd71-9152-4571-b2d9-199101e9d540 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:00.307775 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:00.311958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 261/511] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:18:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:00.312733 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1bd71-9152-4571-b2d9-199101e9d540 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:00.323532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:00.373791 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2b203-65ee-461c-b9f9-569e308a78b6 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:00.406801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88018c9a-d898-4a2c-99bc-34c545f7b337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:00.407494 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88018c9a-d898-4a2c-99bc-34c545f7b337 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:00.407802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88018c9a-d898-4a2c-99bc-34c545f7b337 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:00.408082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88018c9a-d898-4a2c-99bc-34c545f7b337 None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:00.408843 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88018c9a-d898-4a2c-99bc-34c545f7b337 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:00.409175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88018c9a-d898-4a2c-99bc-34c545f7b337 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:00.410688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88018c9a-d898-4a2c-99bc-34c545f7b337 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:00.410929 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1bd71-9152-4571-b2d9-199101e9d540 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:00.432213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 251/512] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Sep 19 10:18:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:00.433054 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88018c9a-d898-4a2c-99bc-34c545f7b337 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:00.456666 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8b935-5227-439c-9c06-41e75bec28e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:00.457329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8b935-5227-439c-9c06-41e75bec28e1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:00.457606 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8b935-5227-439c-9c06-41e75bec28e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:00.457874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8b935-5227-439c-9c06-41e75bec28e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:00.469024 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8b935-5227-439c-9c06-41e75bec28e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:00.470479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8b935-5227-439c-9c06-41e75bec28e1 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:00.484020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-35c8b935-5227-439c-9c06-41e75bec28e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:00.484020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:00.484020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:00.484020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:00.484020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:00.484020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:00.484020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:00.484020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:00.484020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:00.484020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:00.484020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:00.484020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:00.484020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:00.484020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:00.484020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:00.484020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:00.489871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:00.489871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:00.489871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:00.489871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:00.489871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:00.489871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:00.489871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:00.489871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:00.489871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:00.489871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:00.489871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:00.489871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:00.489871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:00.489871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:00.489871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:00.489871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:00.489871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:00.490992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:00.490992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:00.490992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:00.490992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 262/513] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:00.490992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8b935-5227-439c-9c06-41e75bec28e1 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:00.524650 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:00.534683 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88018c9a-d898-4a2c-99bc-34c545f7b337 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:00.544424 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:00.568066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce0ab1-47f8-4de1-8c14-12b2e9e18e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:00.568667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce0ab1-47f8-4de1-8c14-12b2e9e18e2b None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:00.568943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce0ab1-47f8-4de1-8c14-12b2e9e18e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:00.569247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce0ab1-47f8-4de1-8c14-12b2e9e18e2b None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:00.569887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce0ab1-47f8-4de1-8c14-12b2e9e18e2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:00.570186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce0ab1-47f8-4de1-8c14-12b2e9e18e2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:00.571606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce0ab1-47f8-4de1-8c14-12b2e9e18e2b None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:00.583065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 252/514] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:00.583841 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce0ab1-47f8-4de1-8c14-12b2e9e18e2b None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:00.586338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8b935-5227-439c-9c06-41e75bec28e1 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:00.619381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bf590-5c5c-4ef3-94cf-187d91505259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:00.619973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bf590-5c5c-4ef3-94cf-187d91505259 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:00.620245 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bf590-5c5c-4ef3-94cf-187d91505259 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:00.620521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bf590-5c5c-4ef3-94cf-187d91505259 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:00.683095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce0ab1-47f8-4de1-8c14-12b2e9e18e2b None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:00.689243 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841bf590-5c5c-4ef3-94cf-187d91505259 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:00.690743 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841bf590-5c5c-4ef3-94cf-187d91505259 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:00.717079 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24fe08-43cf-4ace-85e6-78a8ce0283f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:00.717705 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24fe08-43cf-4ace-85e6-78a8ce0283f2 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:00.717986 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24fe08-43cf-4ace-85e6-78a8ce0283f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:00.718261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24fe08-43cf-4ace-85e6-78a8ce0283f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:00.730035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24fe08-43cf-4ace-85e6-78a8ce0283f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:00.731504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24fe08-43cf-4ace-85e6-78a8ce0283f2 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:00.745003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:00.745170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-9d24fe08-43cf-4ace-85e6-78a8ce0283f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:00.745170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:00.745170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:00.745170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:00.745170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:00.745170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:00.745170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:00.745170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:00.745170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:00.745170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:00.745170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:00.745170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:00.745170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:00.745170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:00.745170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:00.745170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:00.747194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:00.747194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:00.747194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:00.747194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:00.747194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:00.747194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:00.747194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:00.747194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:00.747194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:00.747194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:00.747194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:00.747194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:00.747194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:00.747194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:00.747194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:00.747194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:00.747194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:00.748074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:00.748074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:00.748074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:00.748074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 253/515] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:00.748074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24fe08-43cf-4ace-85e6-78a8ce0283f2 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:00.754135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:00.790401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 263/516] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:18:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:18:00.791326 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bf590-5c5c-4ef3-94cf-187d91505259 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:00.853631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24fe08-43cf-4ace-85e6-78a8ce0283f2 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:00.886611 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2b314-342d-4a0f-91d7-e64083aeb00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:00.887242 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2b314-342d-4a0f-91d7-e64083aeb00a None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:00.887522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2b314-342d-4a0f-91d7-e64083aeb00a None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:00.887805 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2b314-342d-4a0f-91d7-e64083aeb00a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:00.890493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bf590-5c5c-4ef3-94cf-187d91505259 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:00.924910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f1aaf-1018-4a14-ad9c-9afb040ab4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:00.925539 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f1aaf-1018-4a14-ad9c-9afb040ab4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:00.925820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f1aaf-1018-4a14-ad9c-9afb040ab4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:00.926094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f1aaf-1018-4a14-ad9c-9afb040ab4a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:00.926730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f1aaf-1018-4a14-ad9c-9afb040ab4a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:00.927028 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f1aaf-1018-4a14-ad9c-9afb040ab4a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:00.928397 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f1aaf-1018-4a14-ad9c-9afb040ab4a9 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:00.939341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 264/517] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:00.940044 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f1aaf-1018-4a14-ad9c-9afb040ab4a9 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:00.954607 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2b314-342d-4a0f-91d7-e64083aeb00a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:00.956062 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2b314-342d-4a0f-91d7-e64083aeb00a None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:00.958629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:00.958892 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:01.038327 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f1aaf-1018-4a14-ad9c-9afb040ab4a9 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:01.053409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 254/518] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:18:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:18:01.054148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2b314-342d-4a0f-91d7-e64083aeb00a None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:01.076728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d4c37-c18e-4215-ada4-3ec46fda5362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:01.077366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d4c37-c18e-4215-ada4-3ec46fda5362 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:01.077644 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d4c37-c18e-4215-ada4-3ec46fda5362 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:01.077916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d4c37-c18e-4215-ada4-3ec46fda5362 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:01.078553 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d4c37-c18e-4215-ada4-3ec46fda5362 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:01.079969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d4c37-c18e-4215-ada4-3ec46fda5362 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:01.149993 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2b314-342d-4a0f-91d7-e64083aeb00a None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:01.152230 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:01.155440 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 265/519] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 19 10:18:00 2022] POST /identity/v3/users => generated 247 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:18:01.156265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d4c37-c18e-4215-ada4-3ec46fda5362 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:01.189458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aba8a0-1dfa-4235-b763-b906948dd152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:01.190051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aba8a0-1dfa-4235-b763-b906948dd152 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:01.190339 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aba8a0-1dfa-4235-b763-b906948dd152 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:01.190683 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aba8a0-1dfa-4235-b763-b906948dd152 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:01.191346 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aba8a0-1dfa-4235-b763-b906948dd152 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:01.191708 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aba8a0-1dfa-4235-b763-b906948dd152 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:01.193238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aba8a0-1dfa-4235-b763-b906948dd152 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:01.211993 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 255/520] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:01.211993 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aba8a0-1dfa-4235-b763-b906948dd152 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:01.249511 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:01.280217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d4c37-c18e-4215-ada4-3ec46fda5362 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:01.321276 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad592986-b784-4a0f-9f6c-ed7e8bfdf131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:01.321922 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad592986-b784-4a0f-9f6c-ed7e8bfdf131 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:01.322207 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad592986-b784-4a0f-9f6c-ed7e8bfdf131 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:01.322484 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad592986-b784-4a0f-9f6c-ed7e8bfdf131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:01.322880 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aba8a0-1dfa-4235-b763-b906948dd152 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:01.335560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad592986-b784-4a0f-9f6c-ed7e8bfdf131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:01.337107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad592986-b784-4a0f-9f6c-ed7e8bfdf131 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:01.351909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-ad592986-b784-4a0f-9f6c-ed7e8bfdf131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:01.351909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:01.351909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:01.351909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:01.351909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:01.351909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:01.351909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:01.351909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:01.351909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:01.351909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:01.351909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:01.351909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:01.351909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:01.351909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:01.351909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:01.351909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:01.353003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:01.353003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:01.353003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:01.353003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:01.353003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:01.353003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:01.353003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:01.353003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:01.353003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:01.353003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:01.353003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:01.353003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:01.353003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:01.353003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:01.353003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:01.353003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:01.353003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:01.353939 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:01.353939 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:01.353939 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:01.353939 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 266/521] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:01.354810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad592986-b784-4a0f-9f6c-ed7e8bfdf131 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:01.366403 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c5fcc-6414-4449-a1f5-f18931936331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:01.367131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c5fcc-6414-4449-a1f5-f18931936331 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:01.367766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c5fcc-6414-4449-a1f5-f18931936331 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:01.368085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c5fcc-6414-4449-a1f5-f18931936331 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:01.377523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c5fcc-6414-4449-a1f5-f18931936331 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:01.379000 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c5fcc-6414-4449-a1f5-f18931936331 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:01.393296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-120c5fcc-6414-4449-a1f5-f18931936331 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:01.393296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:01.393296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:01.393296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:01.393296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:01.393296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:01.393296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:01.393296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:01.393296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:01.393296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:01.393296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:01.393296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:01.393296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:01.393296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:01.393296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:01.394507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:01.394507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:01.394507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:01.394507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:01.394507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:01.394507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:01.394507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:01.394507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:01.394507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:01.394507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:01.394507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:01.394507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:01.394507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:01.394507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:01.394507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:01.394507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:01.394507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:01.395570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:01.395570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:01.395570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:01.395570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:01.395570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:01.395570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:01.395570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:01.395570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:01.395570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 256/522] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:01.397840 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c5fcc-6414-4449-a1f5-f18931936331 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:01.426305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:01.470881 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad592986-b784-4a0f-9f6c-ed7e8bfdf131 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:01.498860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:01.507936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8867bd-2dde-4627-b845-bbef39230ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:01.511052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8867bd-2dde-4627-b845-bbef39230ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:01.511052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8867bd-2dde-4627-b845-bbef39230ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:01.511052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8867bd-2dde-4627-b845-bbef39230ea0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:01.512770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c5fcc-6414-4449-a1f5-f18931936331 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:01.521908 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8867bd-2dde-4627-b845-bbef39230ea0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:01.523436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8867bd-2dde-4627-b845-bbef39230ea0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:01.537958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-db8867bd-2dde-4627-b845-bbef39230ea0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:01.537958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:01.537958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:01.537958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:01.537958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:01.537958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:01.537958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:01.537958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:01.537958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:01.537958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:01.537958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:01.537958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:01.537958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:01.537958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:01.537958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:01.537958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:01.539111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:01.539111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:01.539111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:01.539111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:01.539111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:01.539111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:01.539111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:01.539111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:01.539111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:01.539111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:01.539111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:01.539111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:01.539111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:01.539111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:01.539111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:01.539111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:01.539111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:01.540273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:01.540273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:01.540273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:01.544178 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 267/523] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:01.544926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8867bd-2dde-4627-b845-bbef39230ea0 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:01.551025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a2ae6-c412-4c8c-bf82-7f70fc3d91b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:01.551775 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a2ae6-c412-4c8c-bf82-7f70fc3d91b4 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:01.552152 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a2ae6-c412-4c8c-bf82-7f70fc3d91b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:01.552521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a2ae6-c412-4c8c-bf82-7f70fc3d91b4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:01.561182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a2ae6-c412-4c8c-bf82-7f70fc3d91b4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:01.562722 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a2ae6-c412-4c8c-bf82-7f70fc3d91b4 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:01.574066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-2d0a2ae6-c412-4c8c-bf82-7f70fc3d91b4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 10:18:01.574066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:01.574066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:01.574066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:01.574066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:01.574066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:01.574066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:01.574066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:01.574066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:01.574066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:01.574066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:01.574066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:01.574066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:01.574066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:01.574066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:01.574066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:01.575745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:01.575745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:01.575745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:01.575745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:01.575745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:01.575745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:01.575745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:01.575745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:01.575745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:01.575745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:01.575745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:01.575745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:01.575745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:01.575745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:01.575745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:01.575745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:01.575745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:01.575745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:01.576691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:01.576691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:01.576691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:01.576691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:01.576691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:01.576691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:01.576691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:01.576691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:01.576691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 10:18:01.576691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:01.576691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 257/524] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:18:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:01.577799 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a2ae6-c412-4c8c-bf82-7f70fc3d91b4 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:01.656588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8867bd-2dde-4627-b845-bbef39230ea0 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:01.667843 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:01.686672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a2ae6-c412-4c8c-bf82-7f70fc3d91b4 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:01.692932 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446be03-fc55-4331-8fb2-22c1b81ece04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:01.693579 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446be03-fc55-4331-8fb2-22c1b81ece04 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:01.693856 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446be03-fc55-4331-8fb2-22c1b81ece04 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:01.694127 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446be03-fc55-4331-8fb2-22c1b81ece04 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:01.694767 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446be03-fc55-4331-8fb2-22c1b81ece04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:01.695069 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446be03-fc55-4331-8fb2-22c1b81ece04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:01.696424 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446be03-fc55-4331-8fb2-22c1b81ece04 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:01.709896 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 268/525] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:01.710744 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446be03-fc55-4331-8fb2-22c1b81ece04 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:01.727901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec14ed-b76e-4177-bda6-fa21a599225f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:01.728424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec14ed-b76e-4177-bda6-fa21a599225f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:01.728702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec14ed-b76e-4177-bda6-fa21a599225f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:01.728977 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec14ed-b76e-4177-bda6-fa21a599225f None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:01.729888 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec14ed-b76e-4177-bda6-fa21a599225f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:01.730199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec14ed-b76e-4177-bda6-fa21a599225f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:01.732159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec14ed-b76e-4177-bda6-fa21a599225f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:01.755334 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 258/526] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:01.755334 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec14ed-b76e-4177-bda6-fa21a599225f None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:01.756669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:01.821172 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446be03-fc55-4331-8fb2-22c1b81ece04 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:01.859073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7aa432-2972-494a-950d-e4ed669b8251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:01.859972 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7aa432-2972-494a-950d-e4ed669b8251 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:01.860081 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec14ed-b76e-4177-bda6-fa21a599225f None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:01.860532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7aa432-2972-494a-950d-e4ed669b8251 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:01.861011 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7aa432-2972-494a-950d-e4ed669b8251 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:01.861869 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7aa432-2972-494a-950d-e4ed669b8251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:01.862364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7aa432-2972-494a-950d-e4ed669b8251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:01.863813 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7aa432-2972-494a-950d-e4ed669b8251 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:01.880364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 269/527] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:01.880364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7aa432-2972-494a-950d-e4ed669b8251 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:01.901010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c831f0-ba28-43de-ab25-f76cf07b45da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:01.901752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c831f0-ba28-43de-ab25-f76cf07b45da None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:01.902119 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c831f0-ba28-43de-ab25-f76cf07b45da None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:01.902554 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c831f0-ba28-43de-ab25-f76cf07b45da None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:01.903352 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c831f0-ba28-43de-ab25-f76cf07b45da None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:01.903724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c831f0-ba28-43de-ab25-f76cf07b45da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:01.905267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c831f0-ba28-43de-ab25-f76cf07b45da None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:01.925156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 259/528] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Sep 19 10:18:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:01.925156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c831f0-ba28-43de-ab25-f76cf07b45da None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:01.932685 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:01.979173 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:01.983454 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7aa432-2972-494a-950d-e4ed669b8251 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:02.018528 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522dd09-6d9d-4eab-a25c-b74ac654f1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:02.019151 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522dd09-6d9d-4eab-a25c-b74ac654f1ec None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:02.019429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522dd09-6d9d-4eab-a25c-b74ac654f1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:02.019703 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522dd09-6d9d-4eab-a25c-b74ac654f1ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:02.026429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522dd09-6d9d-4eab-a25c-b74ac654f1ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:02.027909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522dd09-6d9d-4eab-a25c-b74ac654f1ec None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:02.039500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-a522dd09-6d9d-4eab-a25c-b74ac654f1ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:02.039500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:02.039500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:02.039500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:02.039500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:02.039500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:02.039500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:02.039500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:02.039500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:02.039500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:02.039500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:02.039500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:02.039500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:02.039500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:02.039500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:02.040522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:02.040522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:02.040522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:02.040522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:02.040522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:02.040522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:02.040522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:02.040522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:02.040522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:02.040522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:02.040522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:02.040522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:02.040522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:02.040522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:02.040522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:02.040522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:02.040522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:02.046762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:02.046762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:02.046762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:02.046762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:02.046762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:02.046762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:02.046762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:02.046762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:02.046762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 270/529] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:02.047328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c831f0-ba28-43de-ab25-f76cf07b45da None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:02.047411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522dd09-6d9d-4eab-a25c-b74ac654f1ec None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:02.083855 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bc458-c4ce-4ef6-92c9-d8eb1d5364ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:02.084463 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bc458-c4ce-4ef6-92c9-d8eb1d5364ea None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:02.084745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bc458-c4ce-4ef6-92c9-d8eb1d5364ea None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:02.085019 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bc458-c4ce-4ef6-92c9-d8eb1d5364ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:02.085772 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bc458-c4ce-4ef6-92c9-d8eb1d5364ea None admin] RBAC: Adding query filter params (role.id=39be941a839d4eb6a2146c561c7a2851, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=b4007794a30247c4a318af62d05e3b92) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:02.086080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bc458-c4ce-4ef6-92c9-d8eb1d5364ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:02.087525 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bc458-c4ce-4ef6-92c9-d8eb1d5364ea None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:02.105105 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 260/530] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:18:01 2022] GET /identity/v3/role_assignments?user.id=b4007794a30247c4a318af62d05e3b92&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=39be941a839d4eb6a2146c561c7a2851 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:02.105105 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bc458-c4ce-4ef6-92c9-d8eb1d5364ea None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:02.174770 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522dd09-6d9d-4eab-a25c-b74ac654f1ec None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:02.186749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:02.218235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918ad44-12c0-493d-a2b3-74ba2e926fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:02.218837 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918ad44-12c0-493d-a2b3-74ba2e926fac None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:02.219181 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918ad44-12c0-493d-a2b3-74ba2e926fac None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:02.219464 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918ad44-12c0-493d-a2b3-74ba2e926fac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:02.227111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918ad44-12c0-493d-a2b3-74ba2e926fac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:02.228624 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918ad44-12c0-493d-a2b3-74ba2e926fac None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:02.231061 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bc458-c4ce-4ef6-92c9-d8eb1d5364ea None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:02.242262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-8918ad44-12c0-493d-a2b3-74ba2e926fac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:02.242262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:02.242262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:02.242262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:02.242262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:02.242262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:02.242262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:02.242262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:02.242262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:02.242262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:02.242262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:02.242262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:02.242262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:02.242262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:02.242262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:02.243375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:02.243375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:02.243375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:02.243375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:02.243375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:02.243375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:02.243375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:02.243375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:02.243375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:02.243375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:02.243375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:02.243375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:02.243375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:02.243375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:02.243375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:02.243375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:02.243375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:02.244517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:02.244517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:02.244517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:02.244517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:02.244517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:02.244517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:02.244517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:02.244517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:02.248913 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 271/531] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:02.250087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918ad44-12c0-493d-a2b3-74ba2e926fac None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:02.272717 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:02.274326 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae5e73-141d-4740-9003-16e119748521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:02.275101 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae5e73-141d-4740-9003-16e119748521 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:02.275570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae5e73-141d-4740-9003-16e119748521 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:02.276343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae5e73-141d-4740-9003-16e119748521 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:02.351972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae5e73-141d-4740-9003-16e119748521 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:02.353778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae5e73-141d-4740-9003-16e119748521 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:02.375557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918ad44-12c0-493d-a2b3-74ba2e926fac None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:02.422636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a33b6-407b-4066-a42e-f03df18943d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:02.423646 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a33b6-407b-4066-a42e-f03df18943d3 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:02.423933 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a33b6-407b-4066-a42e-f03df18943d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:02.424208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a33b6-407b-4066-a42e-f03df18943d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:02.425298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a33b6-407b-4066-a42e-f03df18943d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:02.427339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a33b6-407b-4066-a42e-f03df18943d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:02.427449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:02.427958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a33b6-407b-4066-a42e-f03df18943d3 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:02.456039 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 272/532] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:02.457211 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a33b6-407b-4066-a42e-f03df18943d3 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:02.466429 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 261/533] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:18:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:18:02.467698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae5e73-141d-4740-9003-16e119748521 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:02.565438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:02.575773 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae5e73-141d-4740-9003-16e119748521 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:02.595330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a33b6-407b-4066-a42e-f03df18943d3 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:02.613629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cca76-407a-4e3b-816f-5b8628725856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:02.614581 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cca76-407a-4e3b-816f-5b8628725856 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:02.615063 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cca76-407a-4e3b-816f-5b8628725856 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:02.615790 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cca76-407a-4e3b-816f-5b8628725856 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:02.617416 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cca76-407a-4e3b-816f-5b8628725856 None admin] RBAC: Adding query filter params (role.id=39be941a839d4eb6a2146c561c7a2851, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=e8a10f3332584a90a2231564527078cd) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:02.617928 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cca76-407a-4e3b-816f-5b8628725856 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:02.619688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cca76-407a-4e3b-816f-5b8628725856 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:02.626353 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 262/534] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:18:02 2022] GET /identity/v3/role_assignments?user.id=e8a10f3332584a90a2231564527078cd&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=39be941a839d4eb6a2146c561c7a2851 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:02.627484 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cca76-407a-4e3b-816f-5b8628725856 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:02.635561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b854073-dbfb-44d0-941e-ae1532d95d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:02.636192 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b854073-dbfb-44d0-941e-ae1532d95d07 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:02.636471 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b854073-dbfb-44d0-941e-ae1532d95d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:02.637324 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b854073-dbfb-44d0-941e-ae1532d95d07 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:02.638042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b854073-dbfb-44d0-941e-ae1532d95d07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:02.638373 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b854073-dbfb-44d0-941e-ae1532d95d07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:02.640682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b854073-dbfb-44d0-941e-ae1532d95d07 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:02.658167 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 273/535] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:02.664293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b854073-dbfb-44d0-941e-ae1532d95d07 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:02.666606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:02.804138 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cca76-407a-4e3b-816f-5b8628725856 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:02.857639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b854073-dbfb-44d0-941e-ae1532d95d07 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:02.876388 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de929baf-c043-47e5-a5a2-51f6550161e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:02.877405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de929baf-c043-47e5-a5a2-51f6550161e2 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:02.877889 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de929baf-c043-47e5-a5a2-51f6550161e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:02.878363 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de929baf-c043-47e5-a5a2-51f6550161e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:02.904471 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de929baf-c043-47e5-a5a2-51f6550161e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:02.906522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de929baf-c043-47e5-a5a2-51f6550161e2 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:02.934458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-de929baf-c043-47e5-a5a2-51f6550161e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:02.934458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:02.934458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:02.934458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:02.934458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:02.934458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:02.934458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:02.934458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:02.934458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:02.934458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:02.934458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:02.934458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:02.934458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:02.934458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:02.934458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:02.934458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:02.935449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:02.935449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:02.935449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:02.935449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:02.935449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:02.935449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:02.935449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:02.935449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:02.935449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:02.935449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:02.935449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:02.935449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:02.935449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:02.935449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:02.935449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:02.935449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:02.935449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:02.936319 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:02.936319 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:02.936319 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:02.939002 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017392d7-2cbf-4bd4-a386-405bb2185873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:02.939867 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017392d7-2cbf-4bd4-a386-405bb2185873 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:02.940362 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017392d7-2cbf-4bd4-a386-405bb2185873 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:02.940836 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017392d7-2cbf-4bd4-a386-405bb2185873 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:02.941794 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017392d7-2cbf-4bd4-a386-405bb2185873 None admin] RBAC: Adding query filter params (role.id=39be941a839d4eb6a2146c561c7a2851, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=a40b1ed93a6b4e5fb0e8f907627379a7) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:02.942337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017392d7-2cbf-4bd4-a386-405bb2185873 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:02.943984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017392d7-2cbf-4bd4-a386-405bb2185873 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:02.947308 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 263/536] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:02.951954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de929baf-c043-47e5-a5a2-51f6550161e2 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:02.957978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 274/537] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:18:02 2022] GET /identity/v3/role_assignments?user.id=a40b1ed93a6b4e5fb0e8f907627379a7&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=39be941a839d4eb6a2146c561c7a2851 => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:03.084791 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:03.106504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de929baf-c043-47e5-a5a2-51f6550161e2 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:03.146213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a196a5b-885b-4a7e-ad62-43684ed1cad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:03.147190 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a196a5b-885b-4a7e-ad62-43684ed1cad4 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:03.147676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a196a5b-885b-4a7e-ad62-43684ed1cad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:03.148153 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a196a5b-885b-4a7e-ad62-43684ed1cad4 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:03.149411 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a196a5b-885b-4a7e-ad62-43684ed1cad4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:03.149926 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a196a5b-885b-4a7e-ad62-43684ed1cad4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:03.151503 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a196a5b-885b-4a7e-ad62-43684ed1cad4 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:03.165075 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 264/538] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:03.170665 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017392d7-2cbf-4bd4-a386-405bb2185873 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:03.174718 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:03.277627 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017392d7-2cbf-4bd4-a386-405bb2185873 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:03.313294 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cb670-0e6f-4777-9506-2259b1f62b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:03.314287 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cb670-0e6f-4777-9506-2259b1f62b5d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:03.314770 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cb670-0e6f-4777-9506-2259b1f62b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:03.315265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cb670-0e6f-4777-9506-2259b1f62b5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:03.322446 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8cb670-0e6f-4777-9506-2259b1f62b5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:03.324629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8cb670-0e6f-4777-9506-2259b1f62b5d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:03.336509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-bb8cb670-0e6f-4777-9506-2259b1f62b5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:03.336509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:03.336509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:03.336509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:03.336509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:03.336509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:03.336509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:03.336509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:03.336509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:03.336509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:03.336509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:03.336509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:03.336509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:03.336509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:03.336509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:03.337409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:03.337409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:03.337409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:03.337409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:03.337409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:03.337409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:03.337409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:03.337409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:03.337409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:03.337409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:03.337409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:03.337409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:03.337409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:03.337409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:03.337409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:03.337409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:03.337409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:03.338249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:03.338249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:03.338249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:03.338249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:03.338249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:03.338249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:03.338249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:03.338249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:03.341145 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 275/539] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:03.350136 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a196a5b-885b-4a7e-ad62-43684ed1cad4 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:03.470598 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a196a5b-885b-4a7e-ad62-43684ed1cad4 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:03.517166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82194836-465c-47e2-84ee-a322e59ecdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:03.517788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82194836-465c-47e2-84ee-a322e59ecdd8 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:03.518070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82194836-465c-47e2-84ee-a322e59ecdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:03.518345 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82194836-465c-47e2-84ee-a322e59ecdd8 None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:03.519065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82194836-465c-47e2-84ee-a322e59ecdd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:03.519368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82194836-465c-47e2-84ee-a322e59ecdd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:03.520819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82194836-465c-47e2-84ee-a322e59ecdd8 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:03.538378 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 265/540] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:03.543960 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cb670-0e6f-4777-9506-2259b1f62b5d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:03.626301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:03.628545 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2c87a-42a2-44c4-a3e9-658166ac0934 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:03.628833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2c87a-42a2-44c4-a3e9-658166ac0934 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:03.629162 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2c87a-42a2-44c4-a3e9-658166ac0934 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:03.630503 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 266/541] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:03.635513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc81ef4-5ddc-4ce4-ade2-0b40a002d550 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:03.635795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc81ef4-5ddc-4ce4-ade2-0b40a002d550 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:03.636124 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc81ef4-5ddc-4ce4-ade2-0b40a002d550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:03.643249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:03.656776 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-3cc81ef4-5ddc-4ce4-ade2-0b40a002d550 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:03.658303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-3cc81ef4-5ddc-4ce4-ade2-0b40a002d550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:03.669134 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cb670-0e6f-4777-9506-2259b1f62b5d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:03.711250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f7265-8fcb-4661-8f68-57b5bc8243bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:03.711854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f7265-8fcb-4661-8f68-57b5bc8243bf None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:03.712135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f7265-8fcb-4661-8f68-57b5bc8243bf None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:03.712404 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f7265-8fcb-4661-8f68-57b5bc8243bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:03.713209 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f7265-8fcb-4661-8f68-57b5bc8243bf None admin] RBAC: Adding query filter params (role.id=39be941a839d4eb6a2146c561c7a2851, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=a282e74d177544bbbfba0fdb5e52df60) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:03.713519 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f7265-8fcb-4661-8f68-57b5bc8243bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:03.714973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f7265-8fcb-4661-8f68-57b5bc8243bf None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:03.720728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 276/542] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:18:03 2022] GET /identity/v3/role_assignments?user.id=a282e74d177544bbbfba0fdb5e52df60&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=39be941a839d4eb6a2146c561c7a2851 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:03.739550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 267/543] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:18:04.129914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f7265-8fcb-4661-8f68-57b5bc8243bf None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:04.257677 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f7265-8fcb-4661-8f68-57b5bc8243bf None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:04.299137 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:04.313783 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd48ca1-3d1c-4ada-8407-b92fef7fe874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:04.318672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd48ca1-3d1c-4ada-8407-b92fef7fe874 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:04.318672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd48ca1-3d1c-4ada-8407-b92fef7fe874 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:04.318672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd48ca1-3d1c-4ada-8407-b92fef7fe874 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:04.329942 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd48ca1-3d1c-4ada-8407-b92fef7fe874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:04.334901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-edd48ca1-3d1c-4ada-8407-b92fef7fe874 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:04.334901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:04.334901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:04.334901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:04.334901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:04.334901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:04.334901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:04.334901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:04.334901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:04.334901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:04.334901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:04.334901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:04.334901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:04.334901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:04.334901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:04.334901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:04.335823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:04.335823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:04.335823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:04.335823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:04.335823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:04.335823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:04.335823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:04.335823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:04.335823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:04.335823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:04.335823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:04.335823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:04.335823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:04.335823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:04.335823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:04.335823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:04.336629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:04.336629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:04.336629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:04.336629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:04.336629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:04.336629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:04.336629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:04.336629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:04.336629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:04.336629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:04.336629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:04.336629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:04.336629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 277/544] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:18:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:04.342478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc81ef4-5ddc-4ce4-ade2-0b40a002d550 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:04.491159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc81ef4-5ddc-4ce4-ade2-0b40a002d550 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:04.533306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:04.538492 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe03b49-818a-4d4e-9bbd-d677e60ad12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:04.539110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe03b49-818a-4d4e-9bbd-d677e60ad12f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:04.539392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe03b49-818a-4d4e-9bbd-d677e60ad12f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:04.539667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe03b49-818a-4d4e-9bbd-d677e60ad12f None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:04.540335 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe03b49-818a-4d4e-9bbd-d677e60ad12f None admin] RBAC: Adding query filter params (name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:04.540638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe03b49-818a-4d4e-9bbd-d677e60ad12f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:04.542047 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe03b49-818a-4d4e-9bbd-d677e60ad12f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:04.555223 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 268/545] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:18:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:04.561867 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd48ca1-3d1c-4ada-8407-b92fef7fe874 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:04.712760 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd48ca1-3d1c-4ada-8407-b92fef7fe874 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:04.759749 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a28ed0-4e6a-4d24-a148-6141870f6165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:04.760308 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a28ed0-4e6a-4d24-a148-6141870f6165 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:04.760588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a28ed0-4e6a-4d24-a148-6141870f6165 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:04.760973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a28ed0-4e6a-4d24-a148-6141870f6165 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:04.763875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129aab83-929e-4bcf-b99d-74de72c37ce1 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:04.764165 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129aab83-929e-4bcf-b99d-74de72c37ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:04.764441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129aab83-929e-4bcf-b99d-74de72c37ce1 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:04.767181 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 269/546] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:04.776705 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c66a40-3a05-40fa-b58d-8de5b2f28688 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:04.776990 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c66a40-3a05-40fa-b58d-8de5b2f28688 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:04.777849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c66a40-3a05-40fa-b58d-8de5b2f28688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:04.778655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a28ed0-4e6a-4d24-a148-6141870f6165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:04.780296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a28ed0-4e6a-4d24-a148-6141870f6165 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:04.811080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-54a28ed0-4e6a-4d24-a148-6141870f6165 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:04.811080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:04.811080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:04.811080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:04.811080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:04.811080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:04.811080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:04.811080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:04.811080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:04.811080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:04.811080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:04.811080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:04.811080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:04.811080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:04.811080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:04.811080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:04.812138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:04.812138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:04.812138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:04.812138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:04.812138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:04.812138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:04.812138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:04.812138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:04.812138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:04.812138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:04.812138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:04.812138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:04.812138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:04.812138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:04.812138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:04.812138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:04.812138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:04.812975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:04.812975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:04.812975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:04.815045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-f2c66a40-3a05-40fa-b58d-8de5b2f28688 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:04.817897 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 278/547] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:04.818494 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-f2c66a40-3a05-40fa-b58d-8de5b2f28688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:04.822008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a28ed0-4e6a-4d24-a148-6141870f6165 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:04.890857 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:04.945481 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 270/548] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:18:04.995234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a28ed0-4e6a-4d24-a148-6141870f6165 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:05.046369 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a85ce4-6156-41a8-a93a-eb3c8cf6f8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:05.046898 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a85ce4-6156-41a8-a93a-eb3c8cf6f8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:05.047174 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a85ce4-6156-41a8-a93a-eb3c8cf6f8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:05.047453 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a85ce4-6156-41a8-a93a-eb3c8cf6f8c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:05.048092 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a85ce4-6156-41a8-a93a-eb3c8cf6f8c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:05.048395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a85ce4-6156-41a8-a93a-eb3c8cf6f8c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:05.049786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a85ce4-6156-41a8-a93a-eb3c8cf6f8c0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:05.068164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 279/549] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:05.073513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c66a40-3a05-40fa-b58d-8de5b2f28688 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:05.212987 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c66a40-3a05-40fa-b58d-8de5b2f28688 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:05.240655 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:05.254786 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d99a7-d6a4-4f84-979d-29d08f266afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:05.255417 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d99a7-d6a4-4f84-979d-29d08f266afe None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:05.255692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d99a7-d6a4-4f84-979d-29d08f266afe None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:05.255964 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d99a7-d6a4-4f84-979d-29d08f266afe None admin] PATH_INFO: `/v3/users/swift` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:05.267342 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d99a7-d6a4-4f84-979d-29d08f266afe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:05.268787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d99a7-d6a4-4f84-979d-29d08f266afe None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:05.285567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-bb5d99a7-d6a4-4f84-979d-29d08f266afe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 10:18:05.285567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:05.285567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:05.285567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:05.285567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:05.285567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:05.285567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:05.285567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:05.285567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:05.285567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:05.285567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:05.285567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:05.285567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:05.285567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:05.285567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:05.285567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:05.287742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:05.287742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:05.287742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:05.287742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:05.287742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:05.287742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:05.287742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:05.287742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:05.287742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:05.287742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:05.287742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:05.287742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:05.287742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:05.287742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:05.287742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:05.287742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:05.287742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:05.287742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:05.290045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:05.290045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:05.290045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:05.290045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:05.290045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:05.290045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:05.290045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:05.290045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:05.290045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 10:18:05.290045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:05.290045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 271/550] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:18:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:05.294671 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a85ce4-6156-41a8-a93a-eb3c8cf6f8c0 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:05.323732 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d99a7-d6a4-4f84-979d-29d08f266afe None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:05.441682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:05.446005 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a85ce4-6156-41a8-a93a-eb3c8cf6f8c0 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:05.480361 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d99a7-d6a4-4f84-979d-29d08f266afe None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:05.487482 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea614d-e289-4107-845a-bad172844d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:05.488097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea614d-e289-4107-845a-bad172844d97 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:05.488377 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea614d-e289-4107-845a-bad172844d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:05.488652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea614d-e289-4107-845a-bad172844d97 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:05.489615 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea614d-e289-4107-845a-bad172844d97 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:05.489808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea614d-e289-4107-845a-bad172844d97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:05.491230 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea614d-e289-4107-845a-bad172844d97 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:05.519152 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103e7ed-99e6-48a9-85bc-7a092a303ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:05.520204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103e7ed-99e6-48a9-85bc-7a092a303ebf None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:05.520667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103e7ed-99e6-48a9-85bc-7a092a303ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:05.521166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103e7ed-99e6-48a9-85bc-7a092a303ebf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:05.534766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103e7ed-99e6-48a9-85bc-7a092a303ebf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:05.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-6103e7ed-99e6-48a9-85bc-7a092a303ebf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:05.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:05.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:05.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:05.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:05.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:05.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:05.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:05.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:05.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:05.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:05.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:05.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:05.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:05.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:05.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:05.541800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:05.541800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:05.541800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:05.541800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:05.541800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:05.541800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:05.541800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:05.541800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:05.541800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:05.541800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:05.541800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:05.541800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:05.541800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:05.541800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:05.541800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:05.541800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:05.542629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:05.542629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:05.542629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:05.542629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:05.542629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:05.542629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:05.542629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:05.542629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:05.542629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:05.542629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:05.542629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:05.542629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:05.543299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 280/551] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Sep 19 10:18:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:05.544188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f49c77-ec77-4dad-8144-7326549b919c None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:05.544480 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f49c77-ec77-4dad-8144-7326549b919c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:05.544757 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f49c77-ec77-4dad-8144-7326549b919c None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:05.545775 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 272/552] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:18:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:05.550295 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 281/553] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:05.550448 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103e7ed-99e6-48a9-85bc-7a092a303ebf None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:05.558053 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f49c77-ec77-4dad-8144-7326549b919c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:05.602843 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:05.660505 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103e7ed-99e6-48a9-85bc-7a092a303ebf None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:05.682201 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f49c77-ec77-4dad-8144-7326549b919c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:05.696081 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033b827-20a9-4a77-a19e-2abebd1abe96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:05.696689 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033b827-20a9-4a77-a19e-2abebd1abe96 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:05.696969 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033b827-20a9-4a77-a19e-2abebd1abe96 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:05.697300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033b827-20a9-4a77-a19e-2abebd1abe96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:05.718429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76920895-7e09-4406-a28e-9ab8c1fa325a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:05.719147 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76920895-7e09-4406-a28e-9ab8c1fa325a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:05.719347 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76920895-7e09-4406-a28e-9ab8c1fa325a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:05.719590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76920895-7e09-4406-a28e-9ab8c1fa325a None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:05.720260 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76920895-7e09-4406-a28e-9ab8c1fa325a None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:05.720553 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76920895-7e09-4406-a28e-9ab8c1fa325a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:05.721956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76920895-7e09-4406-a28e-9ab8c1fa325a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:05.730778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 282/554] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:18:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:05.732042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286e281-5c4a-410a-8484-8f45715711fd None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:05.732326 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286e281-5c4a-410a-8484-8f45715711fd None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:05.732595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286e281-5c4a-410a-8484-8f45715711fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:05.749855 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f286e281-5c4a-410a-8484-8f45715711fd None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:05.751034 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f286e281-5c4a-410a-8484-8f45715711fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:05.768019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:05.778412 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033b827-20a9-4a77-a19e-2abebd1abe96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:05.779867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033b827-20a9-4a77-a19e-2abebd1abe96 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:05.828020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 283/555] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:18:05.829681 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8100ea-a1ad-42e8-a7e0-d6d5fe68805d None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:05.829969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8100ea-a1ad-42e8-a7e0-d6d5fe68805d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:05.830244 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8100ea-a1ad-42e8-a7e0-d6d5fe68805d None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:05.833175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 284/556] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:05.834062 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8100ea-a1ad-42e8-a7e0-d6d5fe68805d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:05.838872 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:05.896905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 273/557] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:18:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:18:05.898264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ec7b1-bdff-4adf-a2a9-4cc1a5cce42a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:05.898552 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ec7b1-bdff-4adf-a2a9-4cc1a5cce42a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:05.899099 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ec7b1-bdff-4adf-a2a9-4cc1a5cce42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:05.921682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-513ec7b1-bdff-4adf-a2a9-4cc1a5cce42a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:05.922870 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-513ec7b1-bdff-4adf-a2a9-4cc1a5cce42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:05.951189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8100ea-a1ad-42e8-a7e0-d6d5fe68805d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:05.988065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b17558-4365-47ac-84c5-b6f0bbc1e540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:05.989015 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b17558-4365-47ac-84c5-b6f0bbc1e540 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:05.989466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b17558-4365-47ac-84c5-b6f0bbc1e540 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:05.989891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b17558-4365-47ac-84c5-b6f0bbc1e540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:06.001525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b17558-4365-47ac-84c5-b6f0bbc1e540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:06.003305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b17558-4365-47ac-84c5-b6f0bbc1e540 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:06.009224 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 274/558] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:18:06.011021 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ec7b1-bdff-4adf-a2a9-4cc1a5cce42a None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:06.033951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-f9b17558-4365-47ac-84c5-b6f0bbc1e540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:06.033951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:06.033951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:06.033951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:06.033951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:06.033951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:06.033951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:06.033951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:06.033951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:06.033951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:06.033951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:06.033951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:06.033951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:06.033951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:06.033951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:06.033951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:06.034947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:06.034947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:06.034947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:06.034947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:06.034947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:06.034947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:06.034947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:06.034947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:06.034947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:06.034947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:06.034947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:06.034947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:06.034947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:06.034947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:06.034947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:06.034947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:06.034947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:06.035851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:06.035851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:06.035851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:06.038445 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:06.044334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 285/559] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:06.046007 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b17558-4365-47ac-84c5-b6f0bbc1e540 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:06.127225 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:06.155601 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ec7b1-bdff-4adf-a2a9-4cc1a5cce42a None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:06.193319 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989357e-f1c6-4209-8010-12c554a6d203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:06.194024 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989357e-f1c6-4209-8010-12c554a6d203 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:06.194380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989357e-f1c6-4209-8010-12c554a6d203 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:06.194728 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989357e-f1c6-4209-8010-12c554a6d203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:06.200380 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b17558-4365-47ac-84c5-b6f0bbc1e540 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:06.206720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989357e-f1c6-4209-8010-12c554a6d203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:06.208340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989357e-f1c6-4209-8010-12c554a6d203 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:06.227592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-9989357e-f1c6-4209-8010-12c554a6d203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:06.227592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:06.227592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:06.227592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:06.227592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:06.227592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:06.227592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:06.227592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:06.227592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:06.227592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:06.227592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:06.227592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:06.227592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:06.227592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:06.227592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:06.227592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:06.228628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:06.228628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:06.228628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:06.228628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:06.228628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:06.228628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:06.228628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:06.228628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:06.228628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:06.228628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:06.228628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:06.228628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:06.228628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:06.228628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:06.228628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:06.228628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:06.228628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:06.229577 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:06.229577 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:06.229577 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:06.229577 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 275/560] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:06.230833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989357e-f1c6-4209-8010-12c554a6d203 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:06.251834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e18ad-d236-4b4d-a520-e4d52ec16405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:06.252853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e18ad-d236-4b4d-a520-e4d52ec16405 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:06.253419 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e18ad-d236-4b4d-a520-e4d52ec16405 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:06.253907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e18ad-d236-4b4d-a520-e4d52ec16405 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:06.254957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e18ad-d236-4b4d-a520-e4d52ec16405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:06.255853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e18ad-d236-4b4d-a520-e4d52ec16405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:06.257480 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e18ad-d236-4b4d-a520-e4d52ec16405 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:06.274025 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 286/561] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:06.275091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e18ad-d236-4b4d-a520-e4d52ec16405 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:06.344714 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:06.347885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989357e-f1c6-4209-8010-12c554a6d203 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:06.373323 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:06.382025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e49515-9661-4463-9261-0d164d7e06b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:06.382714 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e49515-9661-4463-9261-0d164d7e06b0 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:06.382995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e49515-9661-4463-9261-0d164d7e06b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:06.383265 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e49515-9661-4463-9261-0d164d7e06b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:06.394048 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e49515-9661-4463-9261-0d164d7e06b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:06.394323 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e18ad-d236-4b4d-a520-e4d52ec16405 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:06.399034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-17e49515-9661-4463-9261-0d164d7e06b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:06.399034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:06.399034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:06.399034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:06.399034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:06.399034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:06.399034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:06.399034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:06.399034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:06.399034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:06.399034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:06.399034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:06.399034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:06.399034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:06.399034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:06.399034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:06.400039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:06.400039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:06.400039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:06.400039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:06.400039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:06.400039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:06.400039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:06.400039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:06.400039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:06.400039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:06.400039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:06.400039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:06.400039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:06.400039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:06.400039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:06.400039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:06.400900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:06.400900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:06.400900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:06.400900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:06.400900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:06.400900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:06.400900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:06.400900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:06.400900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:06.400900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:06.400900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:06.400900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:06.400900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 276/562] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:18:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:06.400900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e49515-9661-4463-9261-0d164d7e06b0 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:06.429133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da9dd2-7a13-4129-8456-6fd2369477d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:06.429737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da9dd2-7a13-4129-8456-6fd2369477d5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:06.430008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da9dd2-7a13-4129-8456-6fd2369477d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:06.430407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da9dd2-7a13-4129-8456-6fd2369477d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:06.430903 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da9dd2-7a13-4129-8456-6fd2369477d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:06.431202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da9dd2-7a13-4129-8456-6fd2369477d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:06.432565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da9dd2-7a13-4129-8456-6fd2369477d5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:06.444607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 287/563] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:06.445787 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4548473-9ca9-4ab3-b2e3-246df950a106 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:06.446057 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4548473-9ca9-4ab3-b2e3-246df950a106 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:06.446597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4548473-9ca9-4ab3-b2e3-246df950a106 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:06.447689 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 288/564] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:06.451987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4548473-9ca9-4ab3-b2e3-246df950a106 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:06.504512 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e49515-9661-4463-9261-0d164d7e06b0 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:06.538338 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac47fed-904d-4f0c-b204-1914f1ee83b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:06.539027 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac47fed-904d-4f0c-b204-1914f1ee83b2 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:06.539309 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac47fed-904d-4f0c-b204-1914f1ee83b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:06.539587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac47fed-904d-4f0c-b204-1914f1ee83b2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:06.546869 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac47fed-904d-4f0c-b204-1914f1ee83b2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:06.548306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac47fed-904d-4f0c-b204-1914f1ee83b2 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:06.551364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4548473-9ca9-4ab3-b2e3-246df950a106 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:06.559831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-fac47fed-904d-4f0c-b204-1914f1ee83b2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 10:18:06.559831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:06.559831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:06.559831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:06.559831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:06.559831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:06.559831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:06.559831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:06.559831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:06.559831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:06.559831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:06.559831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:06.559831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:06.559831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:06.559831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:06.559831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:06.560909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:06.560909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:06.560909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:06.560909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:06.560909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:06.560909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:06.560909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:06.560909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:06.560909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:06.560909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:06.560909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:06.560909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:06.560909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:06.560909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:06.560909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:06.560909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:06.560909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:06.560909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:06.562026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:06.562026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:06.562026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:06.562026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:06.562026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:06.562026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:06.562026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:06.562026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:06.562026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 10:18:06.562026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:06.562026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 277/565] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Sep 19 10:18:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:06.563466 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac47fed-904d-4f0c-b204-1914f1ee83b2 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:06.572078 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:06.587798 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f26320-9b68-4226-9d42-4ff8ecd3f27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:06.588421 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f26320-9b68-4226-9d42-4ff8ecd3f27d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:06.588701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f26320-9b68-4226-9d42-4ff8ecd3f27d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:06.588973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f26320-9b68-4226-9d42-4ff8ecd3f27d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:06.598677 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f26320-9b68-4226-9d42-4ff8ecd3f27d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:06.603730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-23f26320-9b68-4226-9d42-4ff8ecd3f27d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:06.603730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:06.603730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:06.603730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:06.603730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:06.603730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:06.603730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:06.603730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:06.603730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:06.603730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:06.603730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:06.603730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:06.603730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:06.603730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:06.603730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:06.603730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:06.604890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:06.604890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:06.604890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:06.604890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:06.604890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:06.604890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:06.604890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:06.604890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:06.604890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:06.604890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:06.604890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:06.604890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:06.604890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:06.604890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:06.604890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:06.604890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:06.605969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:06.605969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:06.605969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:06.605969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:06.605969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:06.605969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:06.605969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:06.605969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:06.605969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:06.605969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:06.605969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:06.605969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:06.605969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 289/566] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:18:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:06.606837 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f26320-9b68-4226-9d42-4ff8ecd3f27d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:06.613801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:06.666731 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac47fed-904d-4f0c-b204-1914f1ee83b2 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:06.700612 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391537d-ab99-49d6-af89-e330a8214296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:06.701182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391537d-ab99-49d6-af89-e330a8214296 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:06.701466 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391537d-ab99-49d6-af89-e330a8214296 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:06.701739 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391537d-ab99-49d6-af89-e330a8214296 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:06.702389 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391537d-ab99-49d6-af89-e330a8214296 None admin] RBAC: Adding query filter params (name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:06.702690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391537d-ab99-49d6-af89-e330a8214296 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:06.704066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391537d-ab99-49d6-af89-e330a8214296 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:06.704789 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f26320-9b68-4226-9d42-4ff8ecd3f27d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:06.711841 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 278/567] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:18:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:06.712949 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ecdd10-9c7b-4571-a0c4-d365a805b7f4 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:06.713262 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ecdd10-9c7b-4571-a0c4-d365a805b7f4 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:06.713538 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ecdd10-9c7b-4571-a0c4-d365a805b7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:06.731885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-21ecdd10-9c7b-4571-a0c4-d365a805b7f4 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:06.733103 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-21ecdd10-9c7b-4571-a0c4-d365a805b7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:06.738031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2dca05-9f3f-45c0-b891-6d418a17244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:06.738645 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dca05-9f3f-45c0-b891-6d418a17244e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:06.738916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dca05-9f3f-45c0-b891-6d418a17244e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:06.739184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dca05-9f3f-45c0-b891-6d418a17244e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:06.746265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2dca05-9f3f-45c0-b891-6d418a17244e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:06.747699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2dca05-9f3f-45c0-b891-6d418a17244e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:06.759047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-9d2dca05-9f3f-45c0-b891-6d418a17244e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:06.759047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:06.759047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:06.759047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:06.759047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:06.759047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:06.759047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:06.759047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:06.759047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:06.759047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:06.759047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:06.759047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:06.759047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:06.759047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:06.759047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:06.759965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:06.759965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:06.759965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:06.759965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:06.759965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:06.759965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:06.759965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:06.759965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:06.759965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:06.759965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:06.759965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:06.759965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:06.759965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:06.759965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:06.759965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:06.759965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:06.759965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:06.760852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:06.760852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:06.760852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:06.760852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:06.760852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:06.760852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:06.760852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:06.760852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:06.760852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 290/568] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:06.761918 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2dca05-9f3f-45c0-b891-6d418a17244e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:06.801561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:06.804936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 279/569] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:18:06.805813 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ecdd10-9c7b-4571-a0c4-d365a805b7f4 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:06.855747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:06.878634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2dca05-9f3f-45c0-b891-6d418a17244e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:06.913016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68f5c4-db4d-44df-a53c-f3dfad3b75a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:06.913712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68f5c4-db4d-44df-a53c-f3dfad3b75a8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:06.913994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68f5c4-db4d-44df-a53c-f3dfad3b75a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:06.914269 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68f5c4-db4d-44df-a53c-f3dfad3b75a8 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:06.914987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68f5c4-db4d-44df-a53c-f3dfad3b75a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:06.915290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68f5c4-db4d-44df-a53c-f3dfad3b75a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:06.916688 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68f5c4-db4d-44df-a53c-f3dfad3b75a8 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:06.920952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ecdd10-9c7b-4571-a0c4-d365a805b7f4 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:06.939273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 291/570] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Sep 19 10:18:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:06.940327 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68f5c4-db4d-44df-a53c-f3dfad3b75a8 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:06.962665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb73826-2839-429b-8c2d-0a47a552764e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:06.963203 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb73826-2839-429b-8c2d-0a47a552764e None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:06.963482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb73826-2839-429b-8c2d-0a47a552764e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:06.963755 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb73826-2839-429b-8c2d-0a47a552764e None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:06.964547 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb73826-2839-429b-8c2d-0a47a552764e None admin] RBAC: Adding query filter params (name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:06.964868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb73826-2839-429b-8c2d-0a47a552764e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:06.966337 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb73826-2839-429b-8c2d-0a47a552764e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:06.974648 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 280/571] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:18:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:06.975407 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb73826-2839-429b-8c2d-0a47a552764e None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:07.045148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:07.051672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68f5c4-db4d-44df-a53c-f3dfad3b75a8 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:07.078311 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:07.081489 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb73826-2839-429b-8c2d-0a47a552764e None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:07.088296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855cc99f-dfe8-4a71-8255-ac81eec6bfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:07.088923 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cc99f-dfe8-4a71-8255-ac81eec6bfe2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:07.089234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cc99f-dfe8-4a71-8255-ac81eec6bfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:07.089542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cc99f-dfe8-4a71-8255-ac81eec6bfe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:07.099630 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855cc99f-dfe8-4a71-8255-ac81eec6bfe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:07.101070 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855cc99f-dfe8-4a71-8255-ac81eec6bfe2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:07.114835 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9a267-d61e-4c48-b65b-cecbc751a034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:07.115437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9a267-d61e-4c48-b65b-cecbc751a034 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:07.115719 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9a267-d61e-4c48-b65b-cecbc751a034 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:07.115992 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9a267-d61e-4c48-b65b-cecbc751a034 None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:07.116670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9a267-d61e-4c48-b65b-cecbc751a034 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:07.116970 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9a267-d61e-4c48-b65b-cecbc751a034 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:07.117330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-855cc99f-dfe8-4a71-8255-ac81eec6bfe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:07.117330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:07.117330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:07.117330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:07.117330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:07.117330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:07.117330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:07.117330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:07.117330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:07.117330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:07.117330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:07.117330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:07.117330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:07.117330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:07.117330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:07.117330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:07.118407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:07.118407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:07.118407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:07.118407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:07.118407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:07.118407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:07.118407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:07.118407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:07.118407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:07.118407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:07.118407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:07.118407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:07.118407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:07.118407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:07.118407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:07.118407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:07.118407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:07.119366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:07.119366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:07.119366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:07.119542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9a267-d61e-4c48-b65b-cecbc751a034 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:07.119611 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 292/572] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:07.119611 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855cc99f-dfe8-4a71-8255-ac81eec6bfe2 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:07.134309 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 281/573] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:07.135141 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9a267-d61e-4c48-b65b-cecbc751a034 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:07.221530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855cc99f-dfe8-4a71-8255-ac81eec6bfe2 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:07.231468 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9a267-d61e-4c48-b65b-cecbc751a034 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:07.253570 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b45cc7-42c6-4147-8b8e-25bf1db756e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:07.254190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b45cc7-42c6-4147-8b8e-25bf1db756e4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:07.254468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b45cc7-42c6-4147-8b8e-25bf1db756e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:07.254742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b45cc7-42c6-4147-8b8e-25bf1db756e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:07.264493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116224fc-73c2-4649-b7e6-16dd3c60c31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:07.265136 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116224fc-73c2-4649-b7e6-16dd3c60c31e None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:07.265420 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116224fc-73c2-4649-b7e6-16dd3c60c31e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:07.265580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b45cc7-42c6-4147-8b8e-25bf1db756e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:07.265696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116224fc-73c2-4649-b7e6-16dd3c60c31e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:07.267026 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b45cc7-42c6-4147-8b8e-25bf1db756e4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:07.275983 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116224fc-73c2-4649-b7e6-16dd3c60c31e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:07.278030 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116224fc-73c2-4649-b7e6-16dd3c60c31e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:07.281468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-28b45cc7-42c6-4147-8b8e-25bf1db756e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:07.281468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:07.281468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:07.281468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:07.281468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:07.281468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:07.281468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:07.281468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:07.281468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:07.281468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:07.281468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:07.281468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:07.281468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:07.281468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:07.281468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:07.281468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:07.282516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:07.282516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:07.282516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:07.282516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:07.282516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:07.282516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:07.282516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:07.282516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:07.282516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:07.282516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:07.282516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:07.282516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:07.282516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:07.282516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:07.282516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:07.282516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:07.282516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:07.283411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:07.283411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:07.283411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:07.283411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 293/574] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:07.283411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b45cc7-42c6-4147-8b8e-25bf1db756e4 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:07.289349 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:07.292392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-116224fc-73c2-4649-b7e6-16dd3c60c31e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:07.292392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:07.292392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:07.292392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:07.292392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:07.292392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:07.292392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:07.292392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:07.292392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:07.292392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:07.292392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:07.292392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:07.292392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:07.292392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:07.292392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:07.292392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:07.294031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:07.294031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:07.294031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:07.294031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:07.294031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:07.294031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:07.294031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:07.294031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:07.294031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:07.294031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:07.294031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:07.294031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:07.294031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:07.294031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:07.294031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:07.294031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:07.294031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:07.294936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:07.294936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:07.294936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:07.294936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 282/575] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:07.294936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116224fc-73c2-4649-b7e6-16dd3c60c31e None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:07.312786 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:07.385054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b45cc7-42c6-4147-8b8e-25bf1db756e4 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:07.395814 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116224fc-73c2-4649-b7e6-16dd3c60c31e None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:07.421218 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15573b3-3141-4439-a21a-f1d072ce30c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:07.421838 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15573b3-3141-4439-a21a-f1d072ce30c7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:07.422126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15573b3-3141-4439-a21a-f1d072ce30c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:07.422399 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15573b3-3141-4439-a21a-f1d072ce30c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:07.429881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed57c53-27b2-48d4-8860-9bc511ab83ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:07.430560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed57c53-27b2-48d4-8860-9bc511ab83ce None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:07.430849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed57c53-27b2-48d4-8860-9bc511ab83ce None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:07.431126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed57c53-27b2-48d4-8860-9bc511ab83ce None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:07.431762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed57c53-27b2-48d4-8860-9bc511ab83ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:07.432070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed57c53-27b2-48d4-8860-9bc511ab83ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:07.432314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15573b3-3141-4439-a21a-f1d072ce30c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:07.433486 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed57c53-27b2-48d4-8860-9bc511ab83ce None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:07.437566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-f15573b3-3141-4439-a21a-f1d072ce30c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:07.437566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:07.437566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:07.437566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:07.437566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:07.437566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:07.437566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:07.437566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:07.437566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:07.437566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:07.437566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:07.437566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:07.437566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:07.437566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:07.437566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:07.437566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:07.438639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:07.438639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:07.438639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:07.438639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:07.438639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:07.438639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:07.438639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:07.438639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:07.438639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:07.438639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:07.438639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:07.438639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:07.438639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:07.438639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:07.438639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:07.438639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:07.442019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:07.442019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:07.442019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:07.442019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:07.442019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:07.442019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:07.442019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:07.442019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:07.442019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:07.442019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:07.442019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:07.442019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:07.442019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 294/576] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:18:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:07.442019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15573b3-3141-4439-a21a-f1d072ce30c7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:07.450706 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 283/577] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:07.450706 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed57c53-27b2-48d4-8860-9bc511ab83ce None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:07.499569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:07.519360 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:07.537800 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15573b3-3141-4439-a21a-f1d072ce30c7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:07.549238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed57c53-27b2-48d4-8860-9bc511ab83ce None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:07.569911 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044d8ff-afa6-4dfd-b05d-bc5b7848a14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:07.570505 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044d8ff-afa6-4dfd-b05d-bc5b7848a14f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:07.570784 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044d8ff-afa6-4dfd-b05d-bc5b7848a14f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:07.571054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044d8ff-afa6-4dfd-b05d-bc5b7848a14f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:07.571769 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044d8ff-afa6-4dfd-b05d-bc5b7848a14f None admin] RBAC: Adding query filter params (role.id=39be941a839d4eb6a2146c561c7a2851, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=17a8bc6c506c45be897bda23eaab485a) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:07.572308 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044d8ff-afa6-4dfd-b05d-bc5b7848a14f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:07.573507 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044d8ff-afa6-4dfd-b05d-bc5b7848a14f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:07.578683 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 295/578] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:18:07 2022] GET /identity/v3/role_assignments?user.id=17a8bc6c506c45be897bda23eaab485a&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=39be941a839d4eb6a2146c561c7a2851 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:07.579455 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044d8ff-afa6-4dfd-b05d-bc5b7848a14f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:07.582190 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc8bf1-15ab-4363-a7a3-942452452bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:07.582807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc8bf1-15ab-4363-a7a3-942452452bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:07.583082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc8bf1-15ab-4363-a7a3-942452452bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:07.583352 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc8bf1-15ab-4363-a7a3-942452452bc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:07.583998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc8bf1-15ab-4363-a7a3-942452452bc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:07.584299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc8bf1-15ab-4363-a7a3-942452452bc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:07.585732 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc8bf1-15ab-4363-a7a3-942452452bc4 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:07.600708 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 284/579] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:07.601544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc8bf1-15ab-4363-a7a3-942452452bc4 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:07.703210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044d8ff-afa6-4dfd-b05d-bc5b7848a14f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:07.724713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc8bf1-15ab-4363-a7a3-942452452bc4 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:07.734369 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:07.744034 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5118da0-a24a-4719-9ce5-da3bab693101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:07.745427 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5118da0-a24a-4719-9ce5-da3bab693101 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:07.745427 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5118da0-a24a-4719-9ce5-da3bab693101 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:07.745427 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5118da0-a24a-4719-9ce5-da3bab693101 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:07.746828 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5118da0-a24a-4719-9ce5-da3bab693101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:07.747134 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5118da0-a24a-4719-9ce5-da3bab693101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:07.748510 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5118da0-a24a-4719-9ce5-da3bab693101 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:07.761179 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:07.761861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 296/580] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:07.762700 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5118da0-a24a-4719-9ce5-da3bab693101 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:07.770314 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419b1db-99e7-4679-b5f5-38d9a71c5abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:07.770927 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419b1db-99e7-4679-b5f5-38d9a71c5abf None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:07.771207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419b1db-99e7-4679-b5f5-38d9a71c5abf None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:07.771482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419b1db-99e7-4679-b5f5-38d9a71c5abf None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:07.772174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419b1db-99e7-4679-b5f5-38d9a71c5abf None admin] RBAC: Adding query filter params (name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:07.777575 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419b1db-99e7-4679-b5f5-38d9a71c5abf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:07.777575 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419b1db-99e7-4679-b5f5-38d9a71c5abf None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:07.787965 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 285/581] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:18:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:07.787965 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419b1db-99e7-4679-b5f5-38d9a71c5abf None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:07.878986 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5118da0-a24a-4719-9ce5-da3bab693101 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:07.897522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419b1db-99e7-4679-b5f5-38d9a71c5abf None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:07.913670 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f21b67-0521-4b79-846a-fa2875f811dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:07.914303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f21b67-0521-4b79-846a-fa2875f811dd None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:07.914582 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f21b67-0521-4b79-846a-fa2875f811dd None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:07.914853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f21b67-0521-4b79-846a-fa2875f811dd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:07.923135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f21b67-0521-4b79-846a-fa2875f811dd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:07.924581 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f21b67-0521-4b79-846a-fa2875f811dd None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:07.931588 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9510f-34a8-48e0-9ca7-8b908b94fb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:07.932214 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9510f-34a8-48e0-9ca7-8b908b94fb17 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:07.932488 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9510f-34a8-48e0-9ca7-8b908b94fb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:07.932760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9510f-34a8-48e0-9ca7-8b908b94fb17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:07.937647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-42f21b67-0521-4b79-846a-fa2875f811dd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 10:18:07.937647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:07.937647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:07.937647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:07.937647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:07.937647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:07.937647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:07.937647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:07.937647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:07.937647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:07.937647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:07.937647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:07.937647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:07.937647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:07.937647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:07.937647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:07.938580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:07.938580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:07.938580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:07.938580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:07.938580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:07.938580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:07.938580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:07.938580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:07.938580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:07.938580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:07.938580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:07.938580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:07.938580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:07.938580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:07.938580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:07.938580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:07.938580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:07.938580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:07.939548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:07.939548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:07.939548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:07.939548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:07.939548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:07.939548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:07.939548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:07.939548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:07.939548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 10:18:07.939548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:07.941660 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9510f-34a8-48e0-9ca7-8b908b94fb17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:07.943482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9510f-34a8-48e0-9ca7-8b908b94fb17 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:07.943600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 297/582] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:18:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:07.944117 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f21b67-0521-4b79-846a-fa2875f811dd None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:07.954895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-93f9510f-34a8-48e0-9ca7-8b908b94fb17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:07.954895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:07.954895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:07.954895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:07.954895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:07.954895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:07.954895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:07.954895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:07.954895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:07.954895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:07.954895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:07.954895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:07.954895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:07.954895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:07.954895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:07.955822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:07.955822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:07.955822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:07.955822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:07.955822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:07.955822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:07.955822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:07.955822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:07.955822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:07.955822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:07.955822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:07.955822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:07.955822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:07.955822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:07.955822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:07.955822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:07.955822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:07.956729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:07.956729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:07.956729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:07.956729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:07.956729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:07.956729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:07.956729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:07.956729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:07.960707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 286/583] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:07.961469 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9510f-34a8-48e0-9ca7-8b908b94fb17 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:07.994875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:08.016458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:08.057779 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f21b67-0521-4b79-846a-fa2875f811dd None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:08.063850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9510f-34a8-48e0-9ca7-8b908b94fb17 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:08.093663 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df894a-402b-46ca-91dd-3abec58aa745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:08.094545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df894a-402b-46ca-91dd-3abec58aa745 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:08.094971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df894a-402b-46ca-91dd-3abec58aa745 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:08.095387 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df894a-402b-46ca-91dd-3abec58aa745 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:08.099870 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087405b-6d6d-43d2-b04e-65d5d1a3e038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:08.100498 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087405b-6d6d-43d2-b04e-65d5d1a3e038 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:08.100781 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087405b-6d6d-43d2-b04e-65d5d1a3e038 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:08.101086 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087405b-6d6d-43d2-b04e-65d5d1a3e038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:08.103862 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df894a-402b-46ca-91dd-3abec58aa745 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:08.105529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df894a-402b-46ca-91dd-3abec58aa745 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:08.111749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087405b-6d6d-43d2-b04e-65d5d1a3e038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:08.113222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087405b-6d6d-43d2-b04e-65d5d1a3e038 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:08.117234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-e8df894a-402b-46ca-91dd-3abec58aa745 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 19 10:18:08.117234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:08.117234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:08.117234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:08.117234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:08.117234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:08.117234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:08.117234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:08.117234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:08.117234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:08.117234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:08.117234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:08.117234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:08.117234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:08.117234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:08.117234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:08.118333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:08.118333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:08.118333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:08.118333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:08.118333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:08.118333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:08.118333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:08.118333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:08.118333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:08.118333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:08.118333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:08.118333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:08.118333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:08.118333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:08.118333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:08.118333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:08.118333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:08.118333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:08.119485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:08.119485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:08.119485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:08.119485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:08.119485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:08.119485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:08.119485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:08.119485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:08.119485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 19 10:18:08.119485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:08.121843 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 298/584] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:18:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:08.122742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df894a-402b-46ca-91dd-3abec58aa745 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:08.133542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-d087405b-6d6d-43d2-b04e-65d5d1a3e038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:08.133542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:08.133542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:08.133542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:08.133542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:08.133542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:08.133542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:08.133542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:08.133542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:08.133542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:08.133542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:08.133542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:08.133542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:08.133542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:08.133542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:08.133542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:08.134529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:08.134529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:08.134529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:08.134529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:08.134529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:08.134529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:08.134529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:08.134529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:08.134529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:08.134529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:08.134529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:08.134529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:08.134529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:08.134529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:08.134529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:08.134529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:08.134529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:08.135982 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:08.135982 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:08.135982 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:08.135982 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 287/585] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:08.135982 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087405b-6d6d-43d2-b04e-65d5d1a3e038 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:08.239442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df894a-402b-46ca-91dd-3abec58aa745 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:08.239802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087405b-6d6d-43d2-b04e-65d5d1a3e038 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:08.247197 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:08.247920 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:08.275510 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7eda9d-0046-4fac-8dc0-22d46f8dc1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:08.276066 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5ed7b-4f61-44b5-935a-be4a13507200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:08.276671 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5ed7b-4f61-44b5-935a-be4a13507200 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:08.277359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5ed7b-4f61-44b5-935a-be4a13507200 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:08.277639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5ed7b-4f61-44b5-935a-be4a13507200 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:08.278181 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7eda9d-0046-4fac-8dc0-22d46f8dc1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:08.278417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5ed7b-4f61-44b5-935a-be4a13507200 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:08.278726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5ed7b-4f61-44b5-935a-be4a13507200 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:08.278977 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7eda9d-0046-4fac-8dc0-22d46f8dc1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:08.279404 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7eda9d-0046-4fac-8dc0-22d46f8dc1d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:08.280149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5ed7b-4f61-44b5-935a-be4a13507200 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:08.280434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7eda9d-0046-4fac-8dc0-22d46f8dc1d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:08.280893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7eda9d-0046-4fac-8dc0-22d46f8dc1d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:08.282554 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7eda9d-0046-4fac-8dc0-22d46f8dc1d1 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:08.295516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 288/586] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:08.296407 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7eda9d-0046-4fac-8dc0-22d46f8dc1d1 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:08.302746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 299/587] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Sep 19 10:18:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:08.303779 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5ed7b-4f61-44b5-935a-be4a13507200 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:08.406320 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7eda9d-0046-4fac-8dc0-22d46f8dc1d1 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:08.407391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5ed7b-4f61-44b5-935a-be4a13507200 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:08.441763 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794395b-48d4-4511-be5a-db9778c7bbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:08.442323 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eeee9d-59ea-4091-be16-1542c11a28c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:08.442444 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794395b-48d4-4511-be5a-db9778c7bbd2 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:08.442669 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794395b-48d4-4511-be5a-db9778c7bbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:08.442926 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794395b-48d4-4511-be5a-db9778c7bbd2 None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:08.443025 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eeee9d-59ea-4091-be16-1542c11a28c0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:08.443391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eeee9d-59ea-4091-be16-1542c11a28c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:08.443665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794395b-48d4-4511-be5a-db9778c7bbd2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:08.443822 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eeee9d-59ea-4091-be16-1542c11a28c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:08.443972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794395b-48d4-4511-be5a-db9778c7bbd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:08.444570 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eeee9d-59ea-4091-be16-1542c11a28c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:08.444882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eeee9d-59ea-4091-be16-1542c11a28c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:08.445446 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794395b-48d4-4511-be5a-db9778c7bbd2 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:08.446668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eeee9d-59ea-4091-be16-1542c11a28c0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:08.464628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 300/588] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:08.464628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eeee9d-59ea-4091-be16-1542c11a28c0 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:08.473496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 289/589] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Sep 19 10:18:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:08.473496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794395b-48d4-4511-be5a-db9778c7bbd2 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:08.496469 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:08.502290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:08.576063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eeee9d-59ea-4091-be16-1542c11a28c0 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:08.578310 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794395b-48d4-4511-be5a-db9778c7bbd2 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:08.613759 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e53dc-9a9a-4b8e-b56b-a356b730201a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:08.614408 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e53dc-9a9a-4b8e-b56b-a356b730201a None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:08.614684 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e53dc-9a9a-4b8e-b56b-a356b730201a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:08.614832 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d549e7-e8e6-4623-929f-712b39d71c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:08.614946 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e53dc-9a9a-4b8e-b56b-a356b730201a None admin] PATH_INFO: `/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/b4007794a30247c4a318af62d05e3b92/roles/39be941a839d4eb6a2146c561c7a2851` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:08.615938 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d549e7-e8e6-4623-929f-712b39d71c69 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:08.616219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d549e7-e8e6-4623-929f-712b39d71c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:08.616490 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d549e7-e8e6-4623-929f-712b39d71c69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:08.626595 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d549e7-e8e6-4623-929f-712b39d71c69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:08.628241 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d549e7-e8e6-4623-929f-712b39d71c69 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:08.631573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e53dc-9a9a-4b8e-b56b-a356b730201a None admin] RBAC: Authorizing `identity:create_grant(project_id=e45b8fbe79994e7585095328a7ec9260, user_id=b4007794a30247c4a318af62d05e3b92, role_id=39be941a839d4eb6a2146c561c7a2851)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:08.632993 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e53dc-9a9a-4b8e-b56b-a356b730201a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:08.642935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-d0d549e7-e8e6-4623-929f-712b39d71c69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:08.642935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:08.642935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:08.642935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:08.642935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:08.642935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:08.642935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:08.642935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:08.642935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:08.642935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:08.642935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:08.642935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:08.642935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:08.642935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:08.642935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:08.642935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:08.644052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:08.644052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:08.644052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:08.644052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:08.644052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:08.644052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:08.644052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:08.644052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:08.644052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:08.644052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:08.644052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:08.644052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:08.644052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:08.644052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:08.644052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:08.644052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:08.644052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:08.645154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:08.645154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:08.645154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:08.645154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 290/590] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:08.645154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d549e7-e8e6-4623-929f-712b39d71c69 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:08.723729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:08.726775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 301/591] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:18:08 2022] PUT /identity/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/b4007794a30247c4a318af62d05e3b92/roles/39be941a839d4eb6a2146c561c7a2851 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:18:08.736661 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e53dc-9a9a-4b8e-b56b-a356b730201a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:08.751269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d549e7-e8e6-4623-929f-712b39d71c69 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:08.806084 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41091ad-3dee-4601-8d0a-9507c3a37cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:08.806877 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41091ad-3dee-4601-8d0a-9507c3a37cee None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:08.807234 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41091ad-3dee-4601-8d0a-9507c3a37cee None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:08.807866 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41091ad-3dee-4601-8d0a-9507c3a37cee None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:08.818890 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41091ad-3dee-4601-8d0a-9507c3a37cee None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:08.820511 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41091ad-3dee-4601-8d0a-9507c3a37cee None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:08.838968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-c41091ad-3dee-4601-8d0a-9507c3a37cee None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 10:18:08.838968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:08.838968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:08.838968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:08.838968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:08.838968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:08.838968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:08.838968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:08.838968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:08.838968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:08.838968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:08.838968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:08.838968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:08.838968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:08.838968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:08.838968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:08.840108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:08.840108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:08.840108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:08.840108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:08.840108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:08.840108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:08.840108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:08.840108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:08.840108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:08.840108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:08.840108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:08.840108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:08.840108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:08.840108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:08.840108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:08.840108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:08.840108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:08.840108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:08.841036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:08.841036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:08.841036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:08.841036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:08.841036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:08.841036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:08.841036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:08.841036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:08.841036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 10:18:08.841036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:08.841036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 291/592] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:18:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:08.846318 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41091ad-3dee-4601-8d0a-9507c3a37cee None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:08.851998 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:08.885586 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e53dc-9a9a-4b8e-b56b-a356b730201a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:08.931861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d9602-8f61-4c1f-852a-525d00366e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:08.932462 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d9602-8f61-4c1f-852a-525d00366e34 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:08.932852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d9602-8f61-4c1f-852a-525d00366e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:08.933838 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d9602-8f61-4c1f-852a-525d00366e34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:08.947187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d9602-8f61-4c1f-852a-525d00366e34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:08.950179 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d9602-8f61-4c1f-852a-525d00366e34 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:08.965358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-9d6d9602-8f61-4c1f-852a-525d00366e34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:08.965358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:08.965358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:08.965358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:08.965358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:08.965358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:08.965358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:08.965358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:08.965358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:08.965358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:08.965358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:08.965358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:08.965358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:08.965358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:08.965358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:08.965358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:08.966361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:08.966361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:08.966361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:08.966361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:08.966361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:08.966361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:08.966361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:08.966361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:08.966361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:08.966361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:08.966361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:08.966361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:08.966361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:08.966361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:08.966361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:08.966361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:08.966361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:08.967326 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:08.967326 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:08.967326 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:08.967326 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 302/593] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:08.967326 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d9602-8f61-4c1f-852a-525d00366e34 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:08.987615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41091ad-3dee-4601-8d0a-9507c3a37cee None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:09.024957 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83238e-fd09-4d85-a14f-eab87b93ba96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:09.025598 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83238e-fd09-4d85-a14f-eab87b93ba96 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:09.025879 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83238e-fd09-4d85-a14f-eab87b93ba96 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:09.026150 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83238e-fd09-4d85-a14f-eab87b93ba96 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:09.026823 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83238e-fd09-4d85-a14f-eab87b93ba96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:09.027128 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83238e-fd09-4d85-a14f-eab87b93ba96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:09.028501 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83238e-fd09-4d85-a14f-eab87b93ba96 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:09.038641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:09.046760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 292/594] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:09.048028 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83238e-fd09-4d85-a14f-eab87b93ba96 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:09.088005 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d9602-8f61-4c1f-852a-525d00366e34 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:09.116534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:09.128658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b6a82-a26d-4e4d-9903-cf88ee5a211d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:09.129647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b6a82-a26d-4e4d-9903-cf88ee5a211d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:09.130081 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b6a82-a26d-4e4d-9903-cf88ee5a211d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:09.131696 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b6a82-a26d-4e4d-9903-cf88ee5a211d None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:09.131696 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b6a82-a26d-4e4d-9903-cf88ee5a211d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:09.132157 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b6a82-a26d-4e4d-9903-cf88ee5a211d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:09.133935 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b6a82-a26d-4e4d-9903-cf88ee5a211d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:09.158511 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 303/595] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Sep 19 10:18:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:09.159608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b6a82-a26d-4e4d-9903-cf88ee5a211d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:09.174874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83238e-fd09-4d85-a14f-eab87b93ba96 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:09.216364 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4774a55-15ff-4d91-8718-451f38c0ff8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:09.217241 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4774a55-15ff-4d91-8718-451f38c0ff8d None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:09.217824 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4774a55-15ff-4d91-8718-451f38c0ff8d None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:09.217824 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4774a55-15ff-4d91-8718-451f38c0ff8d None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:09.218621 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4774a55-15ff-4d91-8718-451f38c0ff8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:09.218998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4774a55-15ff-4d91-8718-451f38c0ff8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:09.220435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4774a55-15ff-4d91-8718-451f38c0ff8d None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:09.237215 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 293/596] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:09.238452 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4774a55-15ff-4d91-8718-451f38c0ff8d None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:09.297695 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:09.299829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b6a82-a26d-4e4d-9903-cf88ee5a211d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:09.336829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bac7b1-2214-457c-93a4-0e84f64441d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:09.337497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bac7b1-2214-457c-93a4-0e84f64441d0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:09.337780 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bac7b1-2214-457c-93a4-0e84f64441d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:09.338113 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bac7b1-2214-457c-93a4-0e84f64441d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:09.345517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bac7b1-2214-457c-93a4-0e84f64441d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:09.347309 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bac7b1-2214-457c-93a4-0e84f64441d0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:09.350454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4774a55-15ff-4d91-8718-451f38c0ff8d None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:09.360978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-f6bac7b1-2214-457c-93a4-0e84f64441d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:09.360978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:09.360978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:09.360978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:09.360978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:09.360978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:09.360978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:09.360978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:09.360978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:09.360978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:09.360978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:09.360978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:09.360978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:09.360978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:09.360978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:09.361956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:09.361956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:09.361956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:09.361956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:09.361956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:09.361956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:09.361956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:09.361956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:09.361956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:09.361956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:09.361956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:09.361956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:09.361956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:09.361956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:09.361956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:09.361956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:09.361956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:09.363146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:09.363146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:09.363146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:09.363146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:09.363146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:09.363146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:09.363146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:09.363146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:09.363146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 304/597] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:09.363731 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bac7b1-2214-457c-93a4-0e84f64441d0 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:09.400531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba9ff6-d3c5-4b30-9136-d10a15617634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:09.401522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba9ff6-d3c5-4b30-9136-d10a15617634 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:09.402163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba9ff6-d3c5-4b30-9136-d10a15617634 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:09.403524 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba9ff6-d3c5-4b30-9136-d10a15617634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:09.414729 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:09.419264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba9ff6-d3c5-4b30-9136-d10a15617634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:09.421325 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba9ff6-d3c5-4b30-9136-d10a15617634 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:09.441104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-faba9ff6-d3c5-4b30-9136-d10a15617634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:09.441104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:09.441104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:09.441104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:09.441104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:09.441104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:09.441104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:09.441104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:09.441104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:09.441104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:09.441104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:09.441104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:09.441104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:09.441104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:09.441104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:09.441104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:09.442350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:09.442350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:09.442350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:09.442350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:09.442350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:09.442350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:09.442350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:09.442350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:09.442350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:09.442350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:09.442350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:09.442350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:09.442350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:09.442350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:09.442350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:09.442350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:09.442350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:09.443441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:09.443441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:09.443441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:09.451999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 294/598] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:09.453134 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba9ff6-d3c5-4b30-9136-d10a15617634 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:09.480259 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bac7b1-2214-457c-93a4-0e84f64441d0 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:09.524449 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9a4dd-57e0-410e-bd68-2b778ba420da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:09.525264 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9a4dd-57e0-410e-bd68-2b778ba420da None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:09.525473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9a4dd-57e0-410e-bd68-2b778ba420da None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:09.525803 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9a4dd-57e0-410e-bd68-2b778ba420da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:09.533753 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9a4dd-57e0-410e-bd68-2b778ba420da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:09.535263 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9a4dd-57e0-410e-bd68-2b778ba420da None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:09.547958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-f6f9a4dd-57e0-410e-bd68-2b778ba420da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:09.547958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:09.547958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:09.547958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:09.547958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:09.547958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:09.547958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:09.547958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:09.547958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:09.547958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:09.547958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:09.547958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:09.547958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:09.547958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:09.547958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:09.548878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:09.548878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:09.548878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:09.548878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:09.548878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:09.548878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:09.548878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:09.548878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:09.548878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:09.548878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:09.548878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:09.548878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:09.548878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:09.548878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:09.548878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:09.548878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:09.548878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:09.549799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:09.549799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:09.549799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:09.549799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:09.549799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:09.549799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:09.549799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:09.549799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:09.553600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 305/599] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:09.554619 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9a4dd-57e0-410e-bd68-2b778ba420da None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:09.596711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba9ff6-d3c5-4b30-9136-d10a15617634 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:09.609539 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:09.640085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc28d3e-5096-4720-bd93-ab937dc27bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:09.641026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc28d3e-5096-4720-bd93-ab937dc27bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:09.641504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc28d3e-5096-4720-bd93-ab937dc27bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:09.641933 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc28d3e-5096-4720-bd93-ab937dc27bb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:09.642810 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc28d3e-5096-4720-bd93-ab937dc27bb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:09.643277 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc28d3e-5096-4720-bd93-ab937dc27bb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:09.644950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc28d3e-5096-4720-bd93-ab937dc27bb6 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:09.660724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 295/600] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:09.661932 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc28d3e-5096-4720-bd93-ab937dc27bb6 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:09.677653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9a4dd-57e0-410e-bd68-2b778ba420da None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:09.681112 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:09.722590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e808e3-e167-46c7-a124-17e253c38520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:09.723431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e808e3-e167-46c7-a124-17e253c38520 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:09.723978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e808e3-e167-46c7-a124-17e253c38520 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:09.724354 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e808e3-e167-46c7-a124-17e253c38520 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:09.725477 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e808e3-e167-46c7-a124-17e253c38520 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:09.725890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e808e3-e167-46c7-a124-17e253c38520 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:09.727741 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e808e3-e167-46c7-a124-17e253c38520 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:09.743661 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 306/601] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:09.750039 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e808e3-e167-46c7-a124-17e253c38520 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:09.779630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc28d3e-5096-4720-bd93-ab937dc27bb6 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:09.813751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206ef5f-71af-483b-8a15-48f47547a7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:09.814363 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206ef5f-71af-483b-8a15-48f47547a7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:09.814644 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206ef5f-71af-483b-8a15-48f47547a7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:09.814921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206ef5f-71af-483b-8a15-48f47547a7b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:09.815606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206ef5f-71af-483b-8a15-48f47547a7b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:09.816046 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206ef5f-71af-483b-8a15-48f47547a7b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:09.817377 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206ef5f-71af-483b-8a15-48f47547a7b7 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:09.834894 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 296/602] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:09.834894 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206ef5f-71af-483b-8a15-48f47547a7b7 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:09.861705 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:09.874179 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e808e3-e167-46c7-a124-17e253c38520 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:09.909238 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a86d83-5f84-45dc-bba0-aec9b81364f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:09.910186 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a86d83-5f84-45dc-bba0-aec9b81364f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:09.910665 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a86d83-5f84-45dc-bba0-aec9b81364f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:09.911132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a86d83-5f84-45dc-bba0-aec9b81364f5 None admin] PATH_INFO: `/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/e8a10f3332584a90a2231564527078cd/roles/39be941a839d4eb6a2146c561c7a2851` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:09.926389 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a86d83-5f84-45dc-bba0-aec9b81364f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e45b8fbe79994e7585095328a7ec9260, user_id=e8a10f3332584a90a2231564527078cd, role_id=39be941a839d4eb6a2146c561c7a2851)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:09.928067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a86d83-5f84-45dc-bba0-aec9b81364f5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:09.935031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206ef5f-71af-483b-8a15-48f47547a7b7 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:09.968188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714b1df-17fa-4058-81f7-bc544b1d2283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:09.968809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714b1df-17fa-4058-81f7-bc544b1d2283 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:09.969179 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714b1df-17fa-4058-81f7-bc544b1d2283 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:09.969458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714b1df-17fa-4058-81f7-bc544b1d2283 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:09.976031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5714b1df-17fa-4058-81f7-bc544b1d2283 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:09.977506 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5714b1df-17fa-4058-81f7-bc544b1d2283 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:09.989700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-5714b1df-17fa-4058-81f7-bc544b1d2283 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:09.989700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:09.989700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:09.989700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:09.989700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:09.989700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:09.989700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:09.989700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:09.989700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:09.989700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:09.989700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:09.989700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:09.989700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:09.989700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:09.989700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:09.990812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:09.990812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:09.990812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:09.990812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:09.990812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:09.990812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:09.990812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:09.990812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:09.990812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:09.990812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:09.990812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:09.990812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:09.990812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:09.990812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:09.990812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:09.990812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:09.990812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:09.991951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:09.991951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:09.991951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:09.991951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:09.991951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:09.991951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:09.991951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:09.991951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:09.993197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 297/603] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:09.994537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c62a72-8b75-4510-afce-10d6c76b6167 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:09.994873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c62a72-8b75-4510-afce-10d6c76b6167 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:09.995148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c62a72-8b75-4510-afce-10d6c76b6167 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:09.997000 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 298/604] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:09.997691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c62a72-8b75-4510-afce-10d6c76b6167 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:10.021606 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 307/605] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:18:09 2022] PUT /identity/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/e8a10f3332584a90a2231564527078cd/roles/39be941a839d4eb6a2146c561c7a2851 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:18:10.023486 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a86d83-5f84-45dc-bba0-aec9b81364f5 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:10.059782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:10.123660 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:10.142152 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c62a72-8b75-4510-afce-10d6c76b6167 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:10.161226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a86d83-5f84-45dc-bba0-aec9b81364f5 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:10.193771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081233c-cde8-4f25-a4ed-17e6953273ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:10.194993 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081233c-cde8-4f25-a4ed-17e6953273ea None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:10.195423 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081233c-cde8-4f25-a4ed-17e6953273ea None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:10.195865 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081233c-cde8-4f25-a4ed-17e6953273ea None admin] PATH_INFO: `/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/a40b1ed93a6b4e5fb0e8f907627379a7/roles/39be941a839d4eb6a2146c561c7a2851` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:10.202882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75230626-4282-4529-b26e-a3b91600800e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:10.203673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75230626-4282-4529-b26e-a3b91600800e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:10.204066 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75230626-4282-4529-b26e-a3b91600800e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:10.205008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75230626-4282-4529-b26e-a3b91600800e None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:10.206595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75230626-4282-4529-b26e-a3b91600800e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:10.207001 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75230626-4282-4529-b26e-a3b91600800e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:10.208653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75230626-4282-4529-b26e-a3b91600800e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:10.215899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081233c-cde8-4f25-a4ed-17e6953273ea None admin] RBAC: Authorizing `identity:create_grant(project_id=e45b8fbe79994e7585095328a7ec9260, user_id=a40b1ed93a6b4e5fb0e8f907627379a7, role_id=39be941a839d4eb6a2146c561c7a2851)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:10.218219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081233c-cde8-4f25-a4ed-17e6953273ea None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:10.222721 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 308/606] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:10.224343 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4266-ecda-4fbe-ae48-76d1a93fcdac None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:10.224714 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4266-ecda-4fbe-ae48-76d1a93fcdac None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:10.225152 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4266-ecda-4fbe-ae48-76d1a93fcdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:10.244444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-cffa4266-ecda-4fbe-ae48-76d1a93fcdac None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:10.245686 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cffa4266-ecda-4fbe-ae48-76d1a93fcdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:10.315282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 299/607] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:18:09 2022] PUT /identity/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/a40b1ed93a6b4e5fb0e8f907627379a7/roles/39be941a839d4eb6a2146c561c7a2851 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:18:10.321217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081233c-cde8-4f25-a4ed-17e6953273ea None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:10.341275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 309/608] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:18:10.495540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081233c-cde8-4f25-a4ed-17e6953273ea None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:10.546724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831ecbb-c227-4d7d-bd8f-aacba662deaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:10.547388 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831ecbb-c227-4d7d-bd8f-aacba662deaa None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:10.547673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831ecbb-c227-4d7d-bd8f-aacba662deaa None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:10.547951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831ecbb-c227-4d7d-bd8f-aacba662deaa None admin] PATH_INFO: `/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/a282e74d177544bbbfba0fdb5e52df60/roles/39be941a839d4eb6a2146c561c7a2851` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:10.556457 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:10.565289 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831ecbb-c227-4d7d-bd8f-aacba662deaa None admin] RBAC: Authorizing `identity:create_grant(project_id=e45b8fbe79994e7585095328a7ec9260, user_id=a282e74d177544bbbfba0fdb5e52df60, role_id=39be941a839d4eb6a2146c561c7a2851)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:10.566755 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831ecbb-c227-4d7d-bd8f-aacba662deaa None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:10.648127 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa4266-ecda-4fbe-ae48-76d1a93fcdac None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:10.652932 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:10.662529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 300/609] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:18:10 2022] PUT /identity/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/a282e74d177544bbbfba0fdb5e52df60/roles/39be941a839d4eb6a2146c561c7a2851 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:18:10.774931 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa4266-ecda-4fbe-ae48-76d1a93fcdac None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:10.813580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35d629-7326-499a-a9c3-dad14168c32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:10.814277 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35d629-7326-499a-a9c3-dad14168c32d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:10.814559 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35d629-7326-499a-a9c3-dad14168c32d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:10.814832 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35d629-7326-499a-a9c3-dad14168c32d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:10.827492 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d35d629-7326-499a-a9c3-dad14168c32d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:10.832516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-8d35d629-7326-499a-a9c3-dad14168c32d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:10.832516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:10.832516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:10.832516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:10.832516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:10.832516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:10.832516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:10.832516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:10.832516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:10.832516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:10.832516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:10.832516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:10.832516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:10.832516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:10.832516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:10.832516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:10.833504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:10.833504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:10.833504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:10.833504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:10.833504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:10.833504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:10.833504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:10.833504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:10.833504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:10.833504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:10.833504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:10.833504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:10.833504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:10.833504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:10.833504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:10.833504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:10.834367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:10.834367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:10.834367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:10.834367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:10.834367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:10.834367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:10.834367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:10.834367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:10.834367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:10.834367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:10.834367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:10.834367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:10.838639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 310/610] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:18:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:10.840912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831ecbb-c227-4d7d-bd8f-aacba662deaa None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:10.953478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831ecbb-c227-4d7d-bd8f-aacba662deaa None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:10.995013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba101af-e284-4a8b-99f9-3b836352990f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:10.996176 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba101af-e284-4a8b-99f9-3b836352990f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:10.996457 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba101af-e284-4a8b-99f9-3b836352990f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:10.996729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba101af-e284-4a8b-99f9-3b836352990f None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:10.997433 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba101af-e284-4a8b-99f9-3b836352990f None admin] RBAC: Adding query filter params (name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:10.997739 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba101af-e284-4a8b-99f9-3b836352990f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:10.999103 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba101af-e284-4a8b-99f9-3b836352990f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:11.011641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 301/611] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:18:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:11.017501 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35d629-7326-499a-a9c3-dad14168c32d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:11.122939 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:11.177353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35d629-7326-499a-a9c3-dad14168c32d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:11.226535 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db4024-bc4c-4aa0-85fd-b44212100392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:11.227190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db4024-bc4c-4aa0-85fd-b44212100392 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:11.227470 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db4024-bc4c-4aa0-85fd-b44212100392 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:11.227743 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db4024-bc4c-4aa0-85fd-b44212100392 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:11.250144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db4024-bc4c-4aa0-85fd-b44212100392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:11.251600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db4024-bc4c-4aa0-85fd-b44212100392 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:11.269417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-54db4024-bc4c-4aa0-85fd-b44212100392 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:11.269417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:11.269417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:11.269417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:11.269417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:11.269417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:11.269417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:11.269417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:11.269417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:11.269417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:11.269417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:11.269417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:11.269417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:11.269417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:11.269417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:11.269417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:11.270395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:11.270395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:11.270395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:11.270395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:11.270395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:11.270395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:11.270395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:11.270395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:11.270395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:11.270395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:11.270395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:11.270395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:11.270395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:11.270395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:11.270395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:11.270395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:11.270395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:11.271280 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:11.271280 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:11.271280 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:11.271280 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 311/612] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:11.278142 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba101af-e284-4a8b-99f9-3b836352990f None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:11.329348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df80d69-c80e-421a-8bb6-46b87d30e676 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:11.329673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df80d69-c80e-421a-8bb6-46b87d30e676 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:11.329903 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df80d69-c80e-421a-8bb6-46b87d30e676 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:11.331355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 312/613] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:11.339935 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b615a-e612-460b-9fe3-bc05c1c2fffd None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:11.340115 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b615a-e612-460b-9fe3-bc05c1c2fffd None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:11.340312 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b615a-e612-460b-9fe3-bc05c1c2fffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:11.351415 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:11.382250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a53b615a-e612-460b-9fe3-bc05c1c2fffd None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:11.382250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a53b615a-e612-460b-9fe3-bc05c1c2fffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:11.436023 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba101af-e284-4a8b-99f9-3b836352990f None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:11.476132 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a6f46-f0ff-4df9-8db0-0d8509bfb994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:11.476765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a6f46-f0ff-4df9-8db0-0d8509bfb994 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:11.481098 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a6f46-f0ff-4df9-8db0-0d8509bfb994 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:11.481098 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a6f46-f0ff-4df9-8db0-0d8509bfb994 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:11.481098 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a6f46-f0ff-4df9-8db0-0d8509bfb994 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:11.481098 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a6f46-f0ff-4df9-8db0-0d8509bfb994 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:11.481098 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a6f46-f0ff-4df9-8db0-0d8509bfb994 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:11.495877 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 302/614] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:11.506546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a6f46-f0ff-4df9-8db0-0d8509bfb994 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:11.517419 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 313/615] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:18:11.654952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a6f46-f0ff-4df9-8db0-0d8509bfb994 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:11.669859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:11.704144 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df006f4-91be-4037-b824-9f7856235c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:11.704778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df006f4-91be-4037-b824-9f7856235c46 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:11.705078 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df006f4-91be-4037-b824-9f7856235c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:11.705363 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df006f4-91be-4037-b824-9f7856235c46 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:11.717780 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df006f4-91be-4037-b824-9f7856235c46 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:11.719365 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df006f4-91be-4037-b824-9f7856235c46 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:11.733131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-7df006f4-91be-4037-b824-9f7856235c46 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 10:18:11.733131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:11.733131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:11.733131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:11.733131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:11.733131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:11.733131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:11.733131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:11.733131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:11.733131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:11.733131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:11.733131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:11.733131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:11.733131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:11.733131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:11.733131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:11.734115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:11.734115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:11.734115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:11.734115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:11.734115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:11.734115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:11.734115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:11.734115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:11.734115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:11.734115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:11.734115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:11.734115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:11.734115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:11.734115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:11.734115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:11.734115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:11.734115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:11.734115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:11.735065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:11.735065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:11.735065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:11.735065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:11.735065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:11.735065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:11.735065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:11.735065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:11.735065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 10:18:11.735065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:11.735065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 303/616] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:18:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:11.738634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b615a-e612-460b-9fe3-bc05c1c2fffd None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:11.817209 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:11.894553 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b615a-e612-460b-9fe3-bc05c1c2fffd None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:11.941237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53a776-029e-44b3-83fa-e07086450308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:11.941902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53a776-029e-44b3-83fa-e07086450308 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:11.942191 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53a776-029e-44b3-83fa-e07086450308 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:11.942470 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53a776-029e-44b3-83fa-e07086450308 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:11.943204 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53a776-029e-44b3-83fa-e07086450308 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:11.943511 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53a776-029e-44b3-83fa-e07086450308 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:11.944938 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53a776-029e-44b3-83fa-e07086450308 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:11.966013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 314/617] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Sep 19 10:18:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:11.971839 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df006f4-91be-4037-b824-9f7856235c46 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:12.055897 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53a776-029e-44b3-83fa-e07086450308 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:12.084659 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df006f4-91be-4037-b824-9f7856235c46 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:12.170331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05f697-edd3-41d0-862e-c5a1b83c32c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:12.171353 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05f697-edd3-41d0-862e-c5a1b83c32c0 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:12.171840 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05f697-edd3-41d0-862e-c5a1b83c32c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:12.172363 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05f697-edd3-41d0-862e-c5a1b83c32c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:12.201712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53a776-029e-44b3-83fa-e07086450308 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:12.202533 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05f697-edd3-41d0-862e-c5a1b83c32c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:12.204039 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:12.204437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05f697-edd3-41d0-862e-c5a1b83c32c0 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:12.238435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-8b05f697-edd3-41d0-862e-c5a1b83c32c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:12.238435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:12.238435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:12.238435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:12.238435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:12.238435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:12.238435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:12.238435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:12.238435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:12.238435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:12.238435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:12.238435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:12.238435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:12.238435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:12.238435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:12.238435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:12.239459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:12.239459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:12.239459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:12.239459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:12.239459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:12.239459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:12.239459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:12.239459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:12.239459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:12.239459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:12.239459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:12.239459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:12.239459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:12.239459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:12.239459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:12.239459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:12.239459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:12.240516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:12.240516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:12.240516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:12.242638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 304/618] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:12.249053 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05f697-edd3-41d0-862e-c5a1b83c32c0 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:12.250054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009792ba-97a8-4cb0-9ed8-7190b39abe25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:12.250692 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009792ba-97a8-4cb0-9ed8-7190b39abe25 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:12.250967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009792ba-97a8-4cb0-9ed8-7190b39abe25 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:12.251236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009792ba-97a8-4cb0-9ed8-7190b39abe25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:12.272469 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009792ba-97a8-4cb0-9ed8-7190b39abe25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:12.277797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-009792ba-97a8-4cb0-9ed8-7190b39abe25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:12.277797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:12.277797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:12.277797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:12.277797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:12.277797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:12.277797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:12.277797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:12.277797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:12.277797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:12.277797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:12.277797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:12.277797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:12.277797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:12.277797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:12.277797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:12.278723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:12.278723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:12.278723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:12.278723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:12.278723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:12.278723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:12.278723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:12.278723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:12.278723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:12.278723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:12.278723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:12.278723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:12.278723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:12.278723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:12.278723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:12.278723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:12.279563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:12.279563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:12.279563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:12.279563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:12.279563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:12.279563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:12.279563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:12.279563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:12.279563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:12.279563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:12.279563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:12.279563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:12.279563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 315/619] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:18:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:18:12.286455 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009792ba-97a8-4cb0-9ed8-7190b39abe25 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:12.303839 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:12.396762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009792ba-97a8-4cb0-9ed8-7190b39abe25 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:12.441816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6287d67-bbd3-4875-8b6f-4d903972b724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:12.442448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6287d67-bbd3-4875-8b6f-4d903972b724 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:12.442737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6287d67-bbd3-4875-8b6f-4d903972b724 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:12.443013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6287d67-bbd3-4875-8b6f-4d903972b724 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:12.443702 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6287d67-bbd3-4875-8b6f-4d903972b724 None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:12.444230 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6287d67-bbd3-4875-8b6f-4d903972b724 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:12.445482 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6287d67-bbd3-4875-8b6f-4d903972b724 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:12.475410 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 316/620] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:18:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:12.481418 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6287d67-bbd3-4875-8b6f-4d903972b724 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:12.500463 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05f697-edd3-41d0-862e-c5a1b83c32c0 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:12.540207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3b2ac-c59b-4ee4-bb5e-eafbba94bde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:12.541247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3b2ac-c59b-4ee4-bb5e-eafbba94bde6 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:12.541544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3b2ac-c59b-4ee4-bb5e-eafbba94bde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:12.541822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3b2ac-c59b-4ee4-bb5e-eafbba94bde6 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:12.542523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3b2ac-c59b-4ee4-bb5e-eafbba94bde6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:12.545773 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3b2ac-c59b-4ee4-bb5e-eafbba94bde6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:12.545773 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3b2ac-c59b-4ee4-bb5e-eafbba94bde6 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:12.549406 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:12.560633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 305/621] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:12.570257 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3b2ac-c59b-4ee4-bb5e-eafbba94bde6 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:12.661879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6287d67-bbd3-4875-8b6f-4d903972b724 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:12.699022 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78207143-6bec-461d-8642-ab3793e6a6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:12.700004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78207143-6bec-461d-8642-ab3793e6a6ff None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:12.700477 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:12.701038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78207143-6bec-461d-8642-ab3793e6a6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:12.701549 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78207143-6bec-461d-8642-ab3793e6a6ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:12.712672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78207143-6bec-461d-8642-ab3793e6a6ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:12.715025 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78207143-6bec-461d-8642-ab3793e6a6ff None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:12.723166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3b2ac-c59b-4ee4-bb5e-eafbba94bde6 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:12.731655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-78207143-6bec-461d-8642-ab3793e6a6ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:12.731655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:12.731655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:12.731655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:12.731655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:12.731655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:12.731655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:12.731655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:12.731655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:12.731655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:12.731655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:12.731655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:12.731655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:12.731655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:12.731655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:12.731655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:12.732605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:12.732605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:12.732605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:12.732605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:12.732605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:12.732605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:12.732605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:12.732605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:12.732605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:12.732605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:12.732605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:12.732605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:12.732605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:12.732605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:12.732605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:12.732605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:12.732605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:12.733548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:12.733548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:12.733548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:12.740047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 317/622] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:12.741847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef68a8b-7e54-446e-920f-7bcda5253b09 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:12.742338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef68a8b-7e54-446e-920f-7bcda5253b09 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:12.742816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef68a8b-7e54-446e-920f-7bcda5253b09 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:12.744816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 318/623] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:12.745777 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef68a8b-7e54-446e-920f-7bcda5253b09 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:12.765870 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd50fad-e253-4fff-81f7-179ff52255f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:12.766886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd50fad-e253-4fff-81f7-179ff52255f8 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:12.767367 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd50fad-e253-4fff-81f7-179ff52255f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:12.767867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd50fad-e253-4fff-81f7-179ff52255f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:12.775654 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd50fad-e253-4fff-81f7-179ff52255f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:12.777932 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd50fad-e253-4fff-81f7-179ff52255f8 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:12.791034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-ccd50fad-e253-4fff-81f7-179ff52255f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:12.791034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:12.791034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:12.791034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:12.791034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:12.791034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:12.791034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:12.791034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:12.791034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:12.791034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:12.791034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:12.791034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:12.791034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:12.791034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:12.791034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:12.792787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:12.792787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:12.792787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:12.792787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:12.792787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:12.792787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:12.792787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:12.792787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:12.792787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:12.792787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:12.792787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:12.792787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:12.792787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:12.792787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:12.792787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:12.792787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:12.792787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:12.793736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:12.793736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:12.793736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:12.793736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:12.793736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:12.793736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:12.793736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:12.793736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:12.799246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 306/624] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:12.799246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c37bfb-02e6-4b08-871b-e8f9dff261a5 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:12.799246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c37bfb-02e6-4b08-871b-e8f9dff261a5 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:12.799246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c37bfb-02e6-4b08-871b-e8f9dff261a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:12.818075 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-38c37bfb-02e6-4b08-871b-e8f9dff261a5 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:12.819647 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-38c37bfb-02e6-4b08-871b-e8f9dff261a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:12.856207 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:12.882832 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef68a8b-7e54-446e-920f-7bcda5253b09 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:12.894150 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 307/625] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:18:12.895477 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c37bfb-02e6-4b08-871b-e8f9dff261a5 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:12.950357 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0507a4-0be5-4c09-a4f0-32dc079d4fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:12.952069 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0507a4-0be5-4c09-a4f0-32dc079d4fda None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:12.952796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0507a4-0be5-4c09-a4f0-32dc079d4fda None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:12.954844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0507a4-0be5-4c09-a4f0-32dc079d4fda None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:12.956422 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0507a4-0be5-4c09-a4f0-32dc079d4fda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:12.957815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0507a4-0be5-4c09-a4f0-32dc079d4fda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:12.965916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0507a4-0be5-4c09-a4f0-32dc079d4fda None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:12.991868 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 319/626] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:12.998428 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0507a4-0be5-4c09-a4f0-32dc079d4fda None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:13.022972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:13.067165 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c37bfb-02e6-4b08-871b-e8f9dff261a5 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:13.107868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0959136-6e0a-4065-a71a-a15c91c6b57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:13.108637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0959136-6e0a-4065-a71a-a15c91c6b57c None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:13.109141 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0959136-6e0a-4065-a71a-a15c91c6b57c None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:13.109518 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0959136-6e0a-4065-a71a-a15c91c6b57c None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:13.110414 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0959136-6e0a-4065-a71a-a15c91c6b57c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:13.110787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0959136-6e0a-4065-a71a-a15c91c6b57c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:13.112326 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0959136-6e0a-4065-a71a-a15c91c6b57c None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:13.122088 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0507a4-0be5-4c09-a4f0-32dc079d4fda None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:13.130563 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 308/627] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:13.132956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129564b-d4ba-4642-a4d4-176c8215aa12 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:13.133637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129564b-d4ba-4642-a4d4-176c8215aa12 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:13.134378 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129564b-d4ba-4642-a4d4-176c8215aa12 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:13.136231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 309/628] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:13.137743 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129564b-d4ba-4642-a4d4-176c8215aa12 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:13.174408 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3c304-5902-4ba2-a92b-834448d75f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:13.175030 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3c304-5902-4ba2-a92b-834448d75f8a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:13.175301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3c304-5902-4ba2-a92b-834448d75f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:13.175566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3c304-5902-4ba2-a92b-834448d75f8a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:13.186445 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3c304-5902-4ba2-a92b-834448d75f8a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:13.187281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3c304-5902-4ba2-a92b-834448d75f8a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:13.189244 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:13.200516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-c1a3c304-5902-4ba2-a92b-834448d75f8a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 10:18:13.200516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:13.200516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:13.200516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:13.200516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:13.200516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:13.200516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:13.200516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:13.200516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:13.200516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:13.200516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:13.200516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:13.200516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:13.200516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:13.200516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:13.200516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:13.207473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:13.207473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:13.207473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:13.207473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:13.207473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:13.207473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:13.207473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:13.207473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:13.207473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:13.207473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:13.207473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:13.207473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:13.207473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:13.207473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:13.207473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:13.207473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:13.207473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:13.207473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:13.208534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:13.208534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:13.208534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:13.208534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:13.208534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:13.208534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:13.208534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:13.208534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:13.208534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 10:18:13.208534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:13.208534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 320/629] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Sep 19 10:18:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:13.208534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523236ae-e75a-476a-a327-b8b2ee52e216 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:13.208534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523236ae-e75a-476a-a327-b8b2ee52e216 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:13.209290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523236ae-e75a-476a-a327-b8b2ee52e216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:13.226441 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-523236ae-e75a-476a-a327-b8b2ee52e216 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:13.227626 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-523236ae-e75a-476a-a327-b8b2ee52e216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:13.266767 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129564b-d4ba-4642-a4d4-176c8215aa12 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:13.292263 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 321/630] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:18:13.293530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc6ec4-d096-43f2-83f5-160f8656fa0b None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:13.293813 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc6ec4-d096-43f2-83f5-160f8656fa0b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:13.294086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc6ec4-d096-43f2-83f5-160f8656fa0b None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:13.295589 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 322/631] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:13.295698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:13.297841 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc6ec4-d096-43f2-83f5-160f8656fa0b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:13.302749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b1fdf-eeb2-4103-b844-210b68413b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:13.303471 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b1fdf-eeb2-4103-b844-210b68413b19 None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:13.303809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b1fdf-eeb2-4103-b844-210b68413b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:13.304030 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b1fdf-eeb2-4103-b844-210b68413b19 None admin] PATH_INFO: `/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/17a8bc6c506c45be897bda23eaab485a/roles/39be941a839d4eb6a2146c561c7a2851` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:13.319071 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b1fdf-eeb2-4103-b844-210b68413b19 None admin] RBAC: Authorizing `identity:create_grant(project_id=e45b8fbe79994e7585095328a7ec9260, user_id=17a8bc6c506c45be897bda23eaab485a, role_id=39be941a839d4eb6a2146c561c7a2851)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:13.320610 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b1fdf-eeb2-4103-b844-210b68413b19 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:13.401390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc6ec4-d096-43f2-83f5-160f8656fa0b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:13.427469 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:13.428341 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 310/632] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:18:13 2022] PUT /identity/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/17a8bc6c506c45be897bda23eaab485a/roles/39be941a839d4eb6a2146c561c7a2851 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:18:13.429362 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b1fdf-eeb2-4103-b844-210b68413b19 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:13.444643 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531741b-4190-42b6-b513-323d6dc45351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:13.445679 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531741b-4190-42b6-b513-323d6dc45351 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:13.446108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531741b-4190-42b6-b513-323d6dc45351 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:13.446528 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531741b-4190-42b6-b513-323d6dc45351 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:13.447493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531741b-4190-42b6-b513-323d6dc45351 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:13.448093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531741b-4190-42b6-b513-323d6dc45351 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:13.450328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531741b-4190-42b6-b513-323d6dc45351 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:13.477090 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 323/633] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Sep 19 10:18:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:13.477090 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a7cd2-e27d-4dd8-9ee6-f58a8cc5ea85 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:13.477090 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a7cd2-e27d-4dd8-9ee6-f58a8cc5ea85 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:13.477090 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a7cd2-e27d-4dd8-9ee6-f58a8cc5ea85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:13.494968 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-686a7cd2-e27d-4dd8-9ee6-f58a8cc5ea85 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:13.496739 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-686a7cd2-e27d-4dd8-9ee6-f58a8cc5ea85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:13.581771 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 324/634] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:18:13.582579 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a7cd2-e27d-4dd8-9ee6-f58a8cc5ea85 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:13.600801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b1fdf-eeb2-4103-b844-210b68413b19 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:13.650722 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e670e2-6b53-4750-9c11-a2c7a474fc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:13.652102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e670e2-6b53-4750-9c11-a2c7a474fc12 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:13.652864 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e670e2-6b53-4750-9c11-a2c7a474fc12 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:13.653459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e670e2-6b53-4750-9c11-a2c7a474fc12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:13.666588 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e670e2-6b53-4750-9c11-a2c7a474fc12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:13.671409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-37e670e2-6b53-4750-9c11-a2c7a474fc12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:13.671409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:13.671409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:13.671409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:13.671409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:13.671409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:13.671409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:13.671409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:13.671409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:13.671409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:13.671409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:13.671409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:13.671409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:13.671409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:13.671409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:13.671409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:13.672580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:13.672580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:13.672580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:13.672580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:13.672580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:13.672580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:13.672580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:13.672580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:13.672580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:13.672580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:13.672580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:13.672580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:13.672580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:13.672580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:13.672580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:13.672580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:13.673652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:13.673652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:13.673652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:13.673652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:13.673652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:13.673652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:13.673652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:13.673652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:13.673652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:13.673652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:13.673652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:13.673652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:13.673652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 311/635] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:18:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:13.673652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e670e2-6b53-4750-9c11-a2c7a474fc12 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:13.701041 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a7cd2-e27d-4dd8-9ee6-f58a8cc5ea85 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:13.723096 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:13.734583 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:13.742298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f37b7-5951-40cc-96d1-d0f58b2c7670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:13.742979 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f37b7-5951-40cc-96d1-d0f58b2c7670 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:13.743315 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f37b7-5951-40cc-96d1-d0f58b2c7670 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:13.743646 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f37b7-5951-40cc-96d1-d0f58b2c7670 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:13.809162 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e670e2-6b53-4750-9c11-a2c7a474fc12 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:13.838301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f37b7-5951-40cc-96d1-d0f58b2c7670 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:13.840372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f37b7-5951-40cc-96d1-d0f58b2c7670 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:13.856000 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c5065-69d0-4c88-bc8a-22af7cf50d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:13.856626 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c5065-69d0-4c88-bc8a-22af7cf50d6f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:13.856898 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c5065-69d0-4c88-bc8a-22af7cf50d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:13.857197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c5065-69d0-4c88-bc8a-22af7cf50d6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:13.867626 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c5065-69d0-4c88-bc8a-22af7cf50d6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:13.872952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-326c5065-69d0-4c88-bc8a-22af7cf50d6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:13.872952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:13.872952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:13.872952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:13.872952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:13.872952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:13.872952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:13.872952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:13.872952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:13.872952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:13.872952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:13.872952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:13.872952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:13.872952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:13.872952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:13.872952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:13.873908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:13.873908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:13.873908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:13.873908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:13.873908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:13.873908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:13.873908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:13.873908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:13.873908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:13.873908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:13.873908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:13.873908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:13.873908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:13.873908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:13.873908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:13.873908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:13.874762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:13.874762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:13.874762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:13.874762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:13.874762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:13.874762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:13.874762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:13.874762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:13.874762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:13.874762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:13.874762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:13.874762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:13.878845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 312/636] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:18:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:13.879745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c5065-69d0-4c88-bc8a-22af7cf50d6f None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:13.961639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 325/637] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:18:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:18:13.961639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f37b7-5951-40cc-96d1-d0f58b2c7670 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:13.975779 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:13.991543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c5065-69d0-4c88-bc8a-22af7cf50d6f None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:13.996252 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:14.028210 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa6c1d-ce41-4476-9535-487bd89d458a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:14.028818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa6c1d-ce41-4476-9535-487bd89d458a None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:14.029121 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa6c1d-ce41-4476-9535-487bd89d458a None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:14.029398 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa6c1d-ce41-4476-9535-487bd89d458a None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:14.030065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa6c1d-ce41-4476-9535-487bd89d458a None admin] RBAC: Adding query filter params (name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:14.030369 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa6c1d-ce41-4476-9535-487bd89d458a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:14.031749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa6c1d-ce41-4476-9535-487bd89d458a None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:14.040636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 313/638] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:18:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:14.041305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa6c1d-ce41-4476-9535-487bd89d458a None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:14.066449 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f37b7-5951-40cc-96d1-d0f58b2c7670 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:14.109497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd500bd-ecff-4c7e-8c03-957655bc949f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:14.109962 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd500bd-ecff-4c7e-8c03-957655bc949f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:14.110247 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd500bd-ecff-4c7e-8c03-957655bc949f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:14.110510 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd500bd-ecff-4c7e-8c03-957655bc949f None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:14.111169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd500bd-ecff-4c7e-8c03-957655bc949f None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:14.111465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd500bd-ecff-4c7e-8c03-957655bc949f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:14.112813 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd500bd-ecff-4c7e-8c03-957655bc949f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:14.121851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 326/639] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:18:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:14.127049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd500bd-ecff-4c7e-8c03-957655bc949f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:14.160865 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa6c1d-ce41-4476-9535-487bd89d458a None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:14.200272 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01c660-284b-4902-b7f4-03c7d54738bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:14.201107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01c660-284b-4902-b7f4-03c7d54738bc None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:14.201492 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01c660-284b-4902-b7f4-03c7d54738bc None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:14.201867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01c660-284b-4902-b7f4-03c7d54738bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:14.213297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01c660-284b-4902-b7f4-03c7d54738bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:14.218604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-5e01c660-284b-4902-b7f4-03c7d54738bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:14.218604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:14.218604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:14.218604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:14.218604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:14.218604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:14.218604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:14.218604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:14.218604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:14.218604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:14.218604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:14.218604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:14.218604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:14.218604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:14.218604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:14.218604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:14.219849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:14.219849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:14.219849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:14.219849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:14.219849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:14.219849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:14.219849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:14.219849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:14.219849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:14.219849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:14.219849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:14.219849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:14.219849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:14.219849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:14.219849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:14.219849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:14.221093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:14.221093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:14.221093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:14.221093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:14.221093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:14.221093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:14.221093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:14.221093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:14.221093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:14.221093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:14.221093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:14.221093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:14.221950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:14.225951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 314/640] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:18:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:14.226898 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01c660-284b-4902-b7f4-03c7d54738bc None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:14.243203 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd500bd-ecff-4c7e-8c03-957655bc949f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:14.253246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:14.277571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41449042-14aa-4441-abe1-28f1a5e2a10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:14.278325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41449042-14aa-4441-abe1-28f1a5e2a10d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:14.278647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41449042-14aa-4441-abe1-28f1a5e2a10d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:14.278928 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41449042-14aa-4441-abe1-28f1a5e2a10d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:14.290872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41449042-14aa-4441-abe1-28f1a5e2a10d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:14.292252 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41449042-14aa-4441-abe1-28f1a5e2a10d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:14.306853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-41449042-14aa-4441-abe1-28f1a5e2a10d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:14.306853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:14.306853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:14.306853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:14.306853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:14.306853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:14.306853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:14.306853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:14.306853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:14.306853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:14.306853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:14.306853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:14.306853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:14.306853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:14.306853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:14.306853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:14.308564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:14.308564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:14.308564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:14.308564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:14.308564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:14.308564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:14.308564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:14.308564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:14.308564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:14.308564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:14.308564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:14.308564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:14.308564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:14.308564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:14.308564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:14.308564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:14.308564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:14.309546 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:14.309546 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:14.309546 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:14.309546 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 327/641] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:14.309546 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41449042-14aa-4441-abe1-28f1a5e2a10d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:14.331323 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01c660-284b-4902-b7f4-03c7d54738bc None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:14.366104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1fd2f-5333-4b43-9be3-3507d1ebebf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:14.366845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1fd2f-5333-4b43-9be3-3507d1ebebf0 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:14.367127 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1fd2f-5333-4b43-9be3-3507d1ebebf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:14.367749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1fd2f-5333-4b43-9be3-3507d1ebebf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:14.378946 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1fd2f-5333-4b43-9be3-3507d1ebebf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:14.380536 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1fd2f-5333-4b43-9be3-3507d1ebebf0 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:14.396217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-43c1fd2f-5333-4b43-9be3-3507d1ebebf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:14.396217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:14.396217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:14.396217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:14.396217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:14.396217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:14.396217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:14.396217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:14.396217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:14.396217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:14.396217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:14.396217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:14.396217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:14.396217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:14.396217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:14.396217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:14.397462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:14.397462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:14.397462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:14.397462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:14.397462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:14.397462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:14.397462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:14.397462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:14.397462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:14.397462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:14.397462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:14.397462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:14.397462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:14.397462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:14.397462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:14.397462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:14.397462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:14.398896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:14.398896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:14.398896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:14.400261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 315/642] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:14.405694 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1fd2f-5333-4b43-9be3-3507d1ebebf0 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:14.420941 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41449042-14aa-4441-abe1-28f1a5e2a10d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:14.456894 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:14.462986 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a1841-3c1c-4840-8402-c242c479e29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:14.463519 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a1841-3c1c-4840-8402-c242c479e29c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:14.463979 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a1841-3c1c-4840-8402-c242c479e29c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:14.464176 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a1841-3c1c-4840-8402-c242c479e29c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:14.476913 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a1841-3c1c-4840-8402-c242c479e29c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:14.478415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a1841-3c1c-4840-8402-c242c479e29c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:14.488649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:14.494300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-180a1841-3c1c-4840-8402-c242c479e29c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:14.494300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:14.494300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:14.494300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:14.494300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:14.494300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:14.494300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:14.494300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:14.494300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:14.494300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:14.494300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:14.494300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:14.494300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:14.494300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:14.494300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:14.494300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:14.495878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:14.495878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:14.495878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:14.495878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:14.495878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:14.495878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:14.495878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:14.495878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:14.495878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:14.495878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:14.495878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:14.495878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:14.495878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:14.495878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:14.495878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:14.495878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:14.495878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:14.496941 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:14.496941 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:14.496941 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:14.496941 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 328/643] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:14.496941 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a1841-3c1c-4840-8402-c242c479e29c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:14.518063 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1fd2f-5333-4b43-9be3-3507d1ebebf0 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:14.551905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c833ff-2a28-4cf6-8496-b8cae49997ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:14.552919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c833ff-2a28-4cf6-8496-b8cae49997ff None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:14.553723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c833ff-2a28-4cf6-8496-b8cae49997ff None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:14.554429 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c833ff-2a28-4cf6-8496-b8cae49997ff None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:14.555317 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c833ff-2a28-4cf6-8496-b8cae49997ff None admin] RBAC: Adding query filter params (name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:14.555907 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c833ff-2a28-4cf6-8496-b8cae49997ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:14.558346 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c833ff-2a28-4cf6-8496-b8cae49997ff None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:14.569756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 316/644] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:18:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:14.570705 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c833ff-2a28-4cf6-8496-b8cae49997ff None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:14.608367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a1841-3c1c-4840-8402-c242c479e29c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:14.644396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d4b2f-c19b-46dc-a5fb-8b0ee598da72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:14.645266 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d4b2f-c19b-46dc-a5fb-8b0ee598da72 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:14.645545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d4b2f-c19b-46dc-a5fb-8b0ee598da72 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:14.645826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d4b2f-c19b-46dc-a5fb-8b0ee598da72 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:14.646473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d4b2f-c19b-46dc-a5fb-8b0ee598da72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:14.646777 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d4b2f-c19b-46dc-a5fb-8b0ee598da72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:14.648125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d4b2f-c19b-46dc-a5fb-8b0ee598da72 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:14.661549 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 329/645] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:14.662353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d4b2f-c19b-46dc-a5fb-8b0ee598da72 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:14.685538 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c833ff-2a28-4cf6-8496-b8cae49997ff None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:14.718590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:14.720253 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac24535-6bf5-4a94-a497-aefe9aa2283e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:14.720862 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac24535-6bf5-4a94-a497-aefe9aa2283e None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:14.721173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac24535-6bf5-4a94-a497-aefe9aa2283e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:14.721451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac24535-6bf5-4a94-a497-aefe9aa2283e None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:14.722096 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac24535-6bf5-4a94-a497-aefe9aa2283e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:14.722399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac24535-6bf5-4a94-a497-aefe9aa2283e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:14.723776 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac24535-6bf5-4a94-a497-aefe9aa2283e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:14.731699 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:14.736658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 317/646] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:14.740893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac24535-6bf5-4a94-a497-aefe9aa2283e None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:14.784551 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d4b2f-c19b-46dc-a5fb-8b0ee598da72 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:14.826055 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4a7e6-bc92-4008-9d63-fa0ad4b3aaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:14.827005 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4a7e6-bc92-4008-9d63-fa0ad4b3aaf9 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:14.828087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4a7e6-bc92-4008-9d63-fa0ad4b3aaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:14.828559 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4a7e6-bc92-4008-9d63-fa0ad4b3aaf9 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:14.829340 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c4a7e6-bc92-4008-9d63-fa0ad4b3aaf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:14.829715 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c4a7e6-bc92-4008-9d63-fa0ad4b3aaf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:14.831146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c4a7e6-bc92-4008-9d63-fa0ad4b3aaf9 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:14.843032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 330/647] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:14.843831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4a7e6-bc92-4008-9d63-fa0ad4b3aaf9 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:14.854498 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac24535-6bf5-4a94-a497-aefe9aa2283e None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:14.891856 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7106e4-ede4-46da-a555-d2004c9f7c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:14.892614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7106e4-ede4-46da-a555-d2004c9f7c34 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:14.892895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7106e4-ede4-46da-a555-d2004c9f7c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:14.893280 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7106e4-ede4-46da-a555-d2004c9f7c34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:14.903981 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7106e4-ede4-46da-a555-d2004c9f7c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:14.905552 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7106e4-ede4-46da-a555-d2004c9f7c34 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:14.920167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-5b7106e4-ede4-46da-a555-d2004c9f7c34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:14.920167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:14.920167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:14.920167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:14.920167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:14.920167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:14.920167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:14.920167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:14.920167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:14.920167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:14.920167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:14.920167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:14.920167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:14.920167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:14.920167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:14.920167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:14.926051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:14.926051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:14.926051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:14.926051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:14.926051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:14.926051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:14.926051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:14.926051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:14.926051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:14.926051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:14.926051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:14.926051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:14.926051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:14.926051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:14.926051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:14.926051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:14.926051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:14.927221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:14.927221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:14.927221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:14.927221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 318/648] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:14.927221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7106e4-ede4-46da-a555-d2004c9f7c34 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:14.954711 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4a7e6-bc92-4008-9d63-fa0ad4b3aaf9 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:14.956262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:14.979032 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:14.991691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e162dc-6551-425f-b9f6-c93e27f6f1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:14.992577 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e162dc-6551-425f-b9f6-c93e27f6f1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:14.992857 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e162dc-6551-425f-b9f6-c93e27f6f1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:14.993162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e162dc-6551-425f-b9f6-c93e27f6f1a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:15.001033 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e162dc-6551-425f-b9f6-c93e27f6f1a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:15.002577 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e162dc-6551-425f-b9f6-c93e27f6f1a7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:15.014500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-f2e162dc-6551-425f-b9f6-c93e27f6f1a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:15.014500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:15.014500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:15.014500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:15.014500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:15.014500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:15.014500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:15.014500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:15.014500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:15.014500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:15.014500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:15.014500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:15.014500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:15.014500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:15.014500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:15.015612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:15.015612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:15.015612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:15.015612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:15.015612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:15.015612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:15.015612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:15.015612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:15.015612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:15.015612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:15.015612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:15.015612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:15.015612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:15.015612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:15.015612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:15.015612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:15.015612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:15.016964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:15.016964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:15.016964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:15.016964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:15.016964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:15.016964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:15.016964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:15.016964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:15.022344 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 331/649] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:15.022344 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e162dc-6551-425f-b9f6-c93e27f6f1a7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:15.031555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7106e4-ede4-46da-a555-d2004c9f7c34 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:15.068136 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e3159-3ce5-41e8-8a56-c44446350ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:15.068859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e3159-3ce5-41e8-8a56-c44446350ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:15.069326 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e3159-3ce5-41e8-8a56-c44446350ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:15.069672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e3159-3ce5-41e8-8a56-c44446350ba2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:15.079395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e3159-3ce5-41e8-8a56-c44446350ba2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:15.080934 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e3159-3ce5-41e8-8a56-c44446350ba2 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:15.092727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-b49e3159-3ce5-41e8-8a56-c44446350ba2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 10:18:15.092727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:15.092727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:15.092727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:15.092727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:15.092727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:15.092727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:15.092727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:15.092727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:15.092727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:15.092727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:15.092727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:15.092727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:15.092727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:15.092727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:15.092727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:15.098540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:15.098540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:15.098540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:15.098540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:15.098540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:15.098540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:15.098540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:15.098540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:15.098540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:15.098540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:15.098540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:15.098540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:15.098540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:15.098540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:15.098540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:15.098540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:15.098540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:15.098540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:15.099804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:15.099804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:15.099804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:15.099804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:15.099804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:15.099804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:15.099804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:15.099804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:15.099804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 10:18:15.099804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:15.099804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 319/650] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:18:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:15.099804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e3159-3ce5-41e8-8a56-c44446350ba2 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:15.134513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e162dc-6551-425f-b9f6-c93e27f6f1a7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:15.172474 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc356a7-e157-4276-b264-fcc3849c0cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:15.173247 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc356a7-e157-4276-b264-fcc3849c0cec None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:15.173591 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc356a7-e157-4276-b264-fcc3849c0cec None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:15.173939 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc356a7-e157-4276-b264-fcc3849c0cec None admin] PATH_INFO: `/v3/users/glance` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:15.181234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc356a7-e157-4276-b264-fcc3849c0cec None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:15.183258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc356a7-e157-4276-b264-fcc3849c0cec None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:15.195724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-2fc356a7-e157-4276-b264-fcc3849c0cec None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 19 10:18:15.195724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:15.195724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:15.195724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:15.195724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:15.195724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:15.195724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:15.195724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:15.195724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:15.195724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:15.195724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:15.195724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:15.195724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:15.195724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:15.195724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:15.195724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:15.196907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:15.196907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:15.196907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:15.196907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:15.196907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:15.196907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:15.196907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:15.196907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:15.196907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:15.196907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:15.196907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:15.196907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:15.196907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:15.196907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:15.196907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:15.196907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:15.196907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:15.196907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:15.198111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:15.198111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:15.198111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:15.198111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:15.198111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:15.198111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:15.198111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:15.198111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:15.198111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 19 10:18:15.198111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:15.204379 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 332/651] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:18:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:15.205195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc356a7-e157-4276-b264-fcc3849c0cec None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:15.207657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e3159-3ce5-41e8-8a56-c44446350ba2 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:15.218105 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:15.232975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:15.241695 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c518a-341c-439e-829c-e80a205ebbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:15.242306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c518a-341c-439e-829c-e80a205ebbeb None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:15.242585 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c518a-341c-439e-829c-e80a205ebbeb None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:15.242931 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c518a-341c-439e-829c-e80a205ebbeb None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:15.243495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c518a-341c-439e-829c-e80a205ebbeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:15.244361 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c518a-341c-439e-829c-e80a205ebbeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:15.245731 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c518a-341c-439e-829c-e80a205ebbeb None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:15.262355 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 320/652] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:15.262355 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c518a-341c-439e-829c-e80a205ebbeb None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:15.316037 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc356a7-e157-4276-b264-fcc3849c0cec None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:15.354971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c550c-ccb6-4c1e-aae2-7b80db62bcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:15.355581 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c550c-ccb6-4c1e-aae2-7b80db62bcf1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:15.355859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c550c-ccb6-4c1e-aae2-7b80db62bcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:15.356220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c550c-ccb6-4c1e-aae2-7b80db62bcf1 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:15.356815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c550c-ccb6-4c1e-aae2-7b80db62bcf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:15.357142 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c550c-ccb6-4c1e-aae2-7b80db62bcf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:15.358554 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c550c-ccb6-4c1e-aae2-7b80db62bcf1 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:15.367192 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c518a-341c-439e-829c-e80a205ebbeb None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:15.371119 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 333/653] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:15.376758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c550c-ccb6-4c1e-aae2-7b80db62bcf1 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:15.403714 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82b8fe-0c0f-4de1-87d4-6b642a6dff6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:15.404321 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82b8fe-0c0f-4de1-87d4-6b642a6dff6a None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:15.404646 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82b8fe-0c0f-4de1-87d4-6b642a6dff6a None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:15.404919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82b8fe-0c0f-4de1-87d4-6b642a6dff6a None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:15.405659 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82b8fe-0c0f-4de1-87d4-6b642a6dff6a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:15.405958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82b8fe-0c0f-4de1-87d4-6b642a6dff6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:15.407377 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82b8fe-0c0f-4de1-87d4-6b642a6dff6a None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:15.429992 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 321/654] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Sep 19 10:18:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:15.429992 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82b8fe-0c0f-4de1-87d4-6b642a6dff6a None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:15.458485 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:15.469417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:15.487513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c550c-ccb6-4c1e-aae2-7b80db62bcf1 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:15.523847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e175ec1-acf9-47fe-bb23-8c3881740d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:15.524465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e175ec1-acf9-47fe-bb23-8c3881740d57 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:15.524746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e175ec1-acf9-47fe-bb23-8c3881740d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:15.525021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e175ec1-acf9-47fe-bb23-8c3881740d57 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:15.525816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e175ec1-acf9-47fe-bb23-8c3881740d57 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:15.526125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e175ec1-acf9-47fe-bb23-8c3881740d57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:15.529047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e175ec1-acf9-47fe-bb23-8c3881740d57 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:15.538100 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82b8fe-0c0f-4de1-87d4-6b642a6dff6a None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:15.550147 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 334/655] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Sep 19 10:18:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:15.555700 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e175ec1-acf9-47fe-bb23-8c3881740d57 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:15.578269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca783c-140c-4013-858f-d276001580aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:15.578903 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca783c-140c-4013-858f-d276001580aa None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:15.579183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca783c-140c-4013-858f-d276001580aa None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:15.579459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca783c-140c-4013-858f-d276001580aa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:15.586829 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca783c-140c-4013-858f-d276001580aa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:15.588269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca783c-140c-4013-858f-d276001580aa None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:15.601615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-90ca783c-140c-4013-858f-d276001580aa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 10:18:15.601615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:15.601615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:15.601615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:15.601615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:15.601615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:15.601615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:15.601615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:15.601615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:15.601615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:15.601615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:15.601615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:15.601615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:15.601615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:15.601615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:15.601615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:15.607807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:15.607807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:15.607807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:15.607807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:15.607807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:15.607807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:15.607807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:15.607807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:15.607807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:15.607807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:15.607807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:15.607807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:15.607807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:15.607807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:15.607807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:15.607807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:15.607807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:15.607807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:15.609037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:15.609037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:15.609037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:15.609037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:15.609037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:15.609037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:15.609037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:15.609037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:15.609037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 10:18:15.609037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:15.609037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 322/656] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:18:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:15.609037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca783c-140c-4013-858f-d276001580aa None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:15.669587 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e175ec1-acf9-47fe-bb23-8c3881740d57 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:15.695054 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:15.703599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1eb543-589d-4456-a19f-0a3e4d549feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:15.704203 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1eb543-589d-4456-a19f-0a3e4d549feb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:15.704483 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1eb543-589d-4456-a19f-0a3e4d549feb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:15.704759 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1eb543-589d-4456-a19f-0a3e4d549feb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:15.705515 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1eb543-589d-4456-a19f-0a3e4d549feb None admin] RBAC: Adding query filter params (role.id=39be941a839d4eb6a2146c561c7a2851, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=b4007794a30247c4a318af62d05e3b92) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:15.705828 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1eb543-589d-4456-a19f-0a3e4d549feb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:15.707278 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1eb543-589d-4456-a19f-0a3e4d549feb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:15.707730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca783c-140c-4013-858f-d276001580aa None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:15.713505 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 335/657] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:18:15 2022] GET /identity/v3/role_assignments?user.id=b4007794a30247c4a318af62d05e3b92&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=39be941a839d4eb6a2146c561c7a2851 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:15.714783 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1eb543-589d-4456-a19f-0a3e4d549feb None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:15.726936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:15.743940 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5c259-67b6-480f-a297-a336444d3bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:15.749315 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5c259-67b6-480f-a297-a336444d3bdf None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:15.749315 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5c259-67b6-480f-a297-a336444d3bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:15.749315 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5c259-67b6-480f-a297-a336444d3bdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:15.838182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5c259-67b6-480f-a297-a336444d3bdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:15.842444 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5c259-67b6-480f-a297-a336444d3bdf None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:15.847550 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1eb543-589d-4456-a19f-0a3e4d549feb None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:15.889387 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cdfe3-a3e2-48d1-b08d-cf78ac912d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:15.890003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cdfe3-a3e2-48d1-b08d-cf78ac912d31 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:15.890285 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cdfe3-a3e2-48d1-b08d-cf78ac912d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:15.890557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cdfe3-a3e2-48d1-b08d-cf78ac912d31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:15.946438 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:15.961021 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 323/658] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:18:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:18:15.961905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5c259-67b6-480f-a297-a336444d3bdf None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:15.975550 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1cdfe3-a3e2-48d1-b08d-cf78ac912d31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:15.977259 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1cdfe3-a3e2-48d1-b08d-cf78ac912d31 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:15.980685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:16.072999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5c259-67b6-480f-a297-a336444d3bdf None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:16.086224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 336/659] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:18:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:18:16.087487 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb92ec-c21a-4cb7-8f4f-6f5eefe4ab5a None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:16.087769 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb92ec-c21a-4cb7-8f4f-6f5eefe4ab5a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:16.088036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb92ec-c21a-4cb7-8f4f-6f5eefe4ab5a None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:16.090052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 337/660] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:16.094028 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb92ec-c21a-4cb7-8f4f-6f5eefe4ab5a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:16.116394 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8deaba-c9d9-4bc1-96ef-5524024b2814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:16.117004 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8deaba-c9d9-4bc1-96ef-5524024b2814 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:16.117359 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8deaba-c9d9-4bc1-96ef-5524024b2814 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:16.117589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8deaba-c9d9-4bc1-96ef-5524024b2814 None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:16.118309 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8deaba-c9d9-4bc1-96ef-5524024b2814 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:16.118613 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8deaba-c9d9-4bc1-96ef-5524024b2814 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:16.120072 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8deaba-c9d9-4bc1-96ef-5524024b2814 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:16.138853 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 324/661] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Sep 19 10:18:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:16.139710 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8deaba-c9d9-4bc1-96ef-5524024b2814 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:16.188824 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:16.221271 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb92ec-c21a-4cb7-8f4f-6f5eefe4ab5a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:16.224332 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:16.244134 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8deaba-c9d9-4bc1-96ef-5524024b2814 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:16.257566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf4bd66-05c9-420a-b562-1ea9e864aad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:16.258283 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4bd66-05c9-420a-b562-1ea9e864aad1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:16.258694 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4bd66-05c9-420a-b562-1ea9e864aad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:16.259043 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4bd66-05c9-420a-b562-1ea9e864aad1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:16.270196 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf4bd66-05c9-420a-b562-1ea9e864aad1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:16.271713 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf4bd66-05c9-420a-b562-1ea9e864aad1 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:16.282946 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564d614-de17-4a79-9209-b53e0d8b0edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:16.283574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564d614-de17-4a79-9209-b53e0d8b0edf None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:16.283848 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564d614-de17-4a79-9209-b53e0d8b0edf None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:16.284119 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564d614-de17-4a79-9209-b53e0d8b0edf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:16.288565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-ddf4bd66-05c9-420a-b562-1ea9e864aad1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:16.288565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:16.288565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:16.288565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:16.288565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:16.288565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:16.288565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:16.288565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:16.288565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:16.288565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:16.288565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:16.288565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:16.288565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:16.288565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:16.288565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:16.288565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:16.289786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:16.289786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:16.289786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:16.289786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:16.289786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:16.289786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:16.289786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:16.289786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:16.289786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:16.289786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:16.289786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:16.289786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:16.289786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:16.289786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:16.289786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:16.289786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:16.289786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:16.290912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:16.290912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:16.290912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:16.291542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 338/662] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:16.292755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b305ebe-c609-474d-811c-52d168c3f0f2 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:16.293034 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b305ebe-c609-474d-811c-52d168c3f0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:16.293331 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b305ebe-c609-474d-811c-52d168c3f0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:16.299929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564d614-de17-4a79-9209-b53e0d8b0edf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:16.302242 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564d614-de17-4a79-9209-b53e0d8b0edf None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:16.311504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-2b305ebe-c609-474d-811c-52d168c3f0f2 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:16.312704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2b305ebe-c609-474d-811c-52d168c3f0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:16.318275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-6564d614-de17-4a79-9209-b53e0d8b0edf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:16.318275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:16.318275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:16.318275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:16.318275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:16.318275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:16.318275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:16.318275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:16.318275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:16.318275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:16.318275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:16.318275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:16.318275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:16.318275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:16.318275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:16.318275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:16.319219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:16.319219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:16.319219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:16.319219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:16.319219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:16.319219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:16.319219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:16.319219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:16.319219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:16.319219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:16.319219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:16.319219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:16.319219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:16.319219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:16.319219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:16.319219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:16.319219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:16.320146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:16.320146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:16.320146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:16.324874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 325/663] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:16.325285 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564d614-de17-4a79-9209-b53e0d8b0edf None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:16.386682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 339/664] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:18:16.387543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b305ebe-c609-474d-811c-52d168c3f0f2 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:16.452891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:16.456905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564d614-de17-4a79-9209-b53e0d8b0edf None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:16.497733 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:16.501260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab437c96-4932-4513-aa77-08e84083175e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:16.502403 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab437c96-4932-4513-aa77-08e84083175e None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:16.502689 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab437c96-4932-4513-aa77-08e84083175e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:16.502955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab437c96-4932-4513-aa77-08e84083175e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:16.523583 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b305ebe-c609-474d-811c-52d168c3f0f2 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:16.569839 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0a966-e027-4535-a09e-d95db6c7393d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:16.570808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0a966-e027-4535-a09e-d95db6c7393d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:16.571311 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0a966-e027-4535-a09e-d95db6c7393d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:16.572334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0a966-e027-4535-a09e-d95db6c7393d None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:16.574095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a0a966-e027-4535-a09e-d95db6c7393d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:16.574622 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a0a966-e027-4535-a09e-d95db6c7393d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:16.577142 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a0a966-e027-4535-a09e-d95db6c7393d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:16.590076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab437c96-4932-4513-aa77-08e84083175e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:16.591696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab437c96-4932-4513-aa77-08e84083175e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:16.597225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 340/665] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:16.597968 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0a966-e027-4535-a09e-d95db6c7393d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:16.719847 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:16.725151 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0a966-e027-4535-a09e-d95db6c7393d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:16.725371 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 326/666] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:18:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:18:16.726355 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab437c96-4932-4513-aa77-08e84083175e None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:16.763838 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda42805-b999-44ca-9eba-d1150c782ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:16.764624 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda42805-b999-44ca-9eba-d1150c782ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:16.764971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda42805-b999-44ca-9eba-d1150c782ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:16.765334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda42805-b999-44ca-9eba-d1150c782ea5 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:16.766261 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda42805-b999-44ca-9eba-d1150c782ea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:16.766398 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda42805-b999-44ca-9eba-d1150c782ea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:16.767833 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda42805-b999-44ca-9eba-d1150c782ea5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:16.780408 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 341/667] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:16.781358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda42805-b999-44ca-9eba-d1150c782ea5 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:16.790891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:16.835990 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab437c96-4932-4513-aa77-08e84083175e None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:16.870885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fb178-d997-471e-bec3-d7d5e9a1e07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:16.871515 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fb178-d997-471e-bec3-d7d5e9a1e07f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:16.871792 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fb178-d997-471e-bec3-d7d5e9a1e07f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:16.872069 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fb178-d997-471e-bec3-d7d5e9a1e07f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:16.878707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fb178-d997-471e-bec3-d7d5e9a1e07f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:16.880162 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fb178-d997-471e-bec3-d7d5e9a1e07f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:16.892174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-c74fb178-d997-471e-bec3-d7d5e9a1e07f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:16.892174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:16.892174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:16.892174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:16.892174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:16.892174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:16.892174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:16.892174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:16.892174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:16.892174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:16.892174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:16.892174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:16.892174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:16.892174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:16.892174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:16.893139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:16.893139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:16.893139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:16.893139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:16.893139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:16.893139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:16.893139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:16.893139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:16.893139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:16.893139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:16.893139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:16.893139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:16.893139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:16.893139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:16.893139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:16.893139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:16.893139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:16.899246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:16.899246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:16.899246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:16.899246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:16.899246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:16.899246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:16.899246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:16.899246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:16.899714 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda42805-b999-44ca-9eba-d1150c782ea5 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:16.899790 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 327/668] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:16.899790 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fb178-d997-471e-bec3-d7d5e9a1e07f None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:16.938176 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257dd1e-b2ca-46cd-9dcc-fefc2d02a674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:16.939315 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257dd1e-b2ca-46cd-9dcc-fefc2d02a674 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:16.939672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257dd1e-b2ca-46cd-9dcc-fefc2d02a674 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:16.939956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257dd1e-b2ca-46cd-9dcc-fefc2d02a674 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:16.951261 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257dd1e-b2ca-46cd-9dcc-fefc2d02a674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:16.956164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-7257dd1e-b2ca-46cd-9dcc-fefc2d02a674 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:16.956164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:16.956164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:16.956164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:16.956164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:16.956164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:16.956164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:16.956164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:16.956164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:16.956164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:16.956164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:16.956164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:16.956164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:16.956164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:16.956164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:16.956164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:16.957122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:16.957122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:16.957122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:16.957122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:16.957122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:16.957122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:16.957122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:16.957122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:16.957122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:16.957122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:16.957122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:16.957122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:16.957122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:16.957122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:16.957122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:16.957122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:16.958003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:16.958003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:16.958003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:16.958003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:16.958003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:16.958003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:16.958003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:16.958003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:16.958003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:16.958003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:16.958003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:16.958003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:16.958003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 342/669] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:18:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:16.958743 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257dd1e-b2ca-46cd-9dcc-fefc2d02a674 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:16.960902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:17.007817 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fb178-d997-471e-bec3-d7d5e9a1e07f None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:17.032030 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:17.044215 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e51983-89e6-499e-a960-b010fb5851f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:17.044891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e51983-89e6-499e-a960-b010fb5851f3 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:17.045199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e51983-89e6-499e-a960-b010fb5851f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:17.045474 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e51983-89e6-499e-a960-b010fb5851f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:17.055539 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e51983-89e6-499e-a960-b010fb5851f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:17.058334 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e51983-89e6-499e-a960-b010fb5851f3 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:17.069846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257dd1e-b2ca-46cd-9dcc-fefc2d02a674 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:17.073137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-d7e51983-89e6-499e-a960-b010fb5851f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:17.073137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:17.073137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:17.073137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:17.073137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:17.073137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:17.073137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:17.073137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:17.073137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:17.073137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:17.073137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:17.073137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:17.073137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:17.073137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:17.073137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:17.073137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:17.074301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:17.074301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:17.074301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:17.074301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:17.074301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:17.074301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:17.074301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:17.074301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:17.074301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:17.074301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:17.074301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:17.074301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:17.074301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:17.074301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:17.074301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:17.074301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:17.074301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:17.075400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:17.075400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:17.075400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:17.078875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 328/670] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:17.079557 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e51983-89e6-499e-a960-b010fb5851f3 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:17.110427 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c32d0-d738-49d9-817f-ab76f8059932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:17.111130 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c32d0-d738-49d9-817f-ab76f8059932 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:17.111482 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c32d0-d738-49d9-817f-ab76f8059932 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:17.111827 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c32d0-d738-49d9-817f-ab76f8059932 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:17.119380 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c32d0-d738-49d9-817f-ab76f8059932 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:17.121032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c32d0-d738-49d9-817f-ab76f8059932 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:17.132847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-853c32d0-d738-49d9-817f-ab76f8059932 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:17.132847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:17.132847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:17.132847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:17.132847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:17.132847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:17.132847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:17.132847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:17.132847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:17.132847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:17.132847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:17.132847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:17.132847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:17.132847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:17.132847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:17.134010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:17.134010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:17.134010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:17.134010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:17.134010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:17.134010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:17.134010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:17.134010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:17.134010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:17.134010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:17.134010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:17.134010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:17.134010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:17.134010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:17.134010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:17.134010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:17.134010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:17.135229 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:17.135229 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:17.135229 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:17.135229 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:17.135229 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:17.135229 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:17.135229 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:17.135229 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:17.135929 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 343/671] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:17.136704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c32d0-d738-49d9-817f-ab76f8059932 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:17.184392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e51983-89e6-499e-a960-b010fb5851f3 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:17.194353 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:17.217871 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e19a18-e842-4635-a907-2b48923faff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:17.218480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e19a18-e842-4635-a907-2b48923faff4 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:17.218761 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e19a18-e842-4635-a907-2b48923faff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:17.219039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e19a18-e842-4635-a907-2b48923faff4 None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:17.219718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e19a18-e842-4635-a907-2b48923faff4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:17.220034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e19a18-e842-4635-a907-2b48923faff4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:17.221549 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e19a18-e842-4635-a907-2b48923faff4 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:17.238887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 329/672] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:17.238887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e19a18-e842-4635-a907-2b48923faff4 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:17.248369 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c32d0-d738-49d9-817f-ab76f8059932 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:17.274464 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:17.286132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada125c-bdf0-4887-bd85-796f877b45a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:17.286132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada125c-bdf0-4887-bd85-796f877b45a0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:17.286132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada125c-bdf0-4887-bd85-796f877b45a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:17.286132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada125c-bdf0-4887-bd85-796f877b45a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:17.286541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ada125c-bdf0-4887-bd85-796f877b45a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:17.286541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ada125c-bdf0-4887-bd85-796f877b45a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:17.286704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ada125c-bdf0-4887-bd85-796f877b45a0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:17.301345 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 344/673] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:18:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:17.302530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada125c-bdf0-4887-bd85-796f877b45a0 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:17.341292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e19a18-e842-4635-a907-2b48923faff4 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:17.375236 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f9270-f5ee-4ec2-b12d-1a7e28863bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:17.375945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f9270-f5ee-4ec2-b12d-1a7e28863bff None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:17.376307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f9270-f5ee-4ec2-b12d-1a7e28863bff None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:17.376589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f9270-f5ee-4ec2-b12d-1a7e28863bff None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:17.377261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f9270-f5ee-4ec2-b12d-1a7e28863bff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:17.377569 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f9270-f5ee-4ec2-b12d-1a7e28863bff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:17.378936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f9270-f5ee-4ec2-b12d-1a7e28863bff None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:17.394698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 330/674] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:17.394698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f9270-f5ee-4ec2-b12d-1a7e28863bff None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:17.419743 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada125c-bdf0-4887-bd85-796f877b45a0 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:17.425571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:17.459395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de441382-a8f3-4ab2-b4d5-3450124a4180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:17.460098 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de441382-a8f3-4ab2-b4d5-3450124a4180 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:17.460389 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de441382-a8f3-4ab2-b4d5-3450124a4180 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:17.460709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de441382-a8f3-4ab2-b4d5-3450124a4180 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:17.461439 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de441382-a8f3-4ab2-b4d5-3450124a4180 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:17.461753 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de441382-a8f3-4ab2-b4d5-3450124a4180 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:17.463161 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de441382-a8f3-4ab2-b4d5-3450124a4180 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:17.475346 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 345/675] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:17.476297 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de441382-a8f3-4ab2-b4d5-3450124a4180 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:17.502870 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f9270-f5ee-4ec2-b12d-1a7e28863bff None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:17.531748 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:17.541000 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16ef6b-3dd5-49f8-91cf-89cc3067234f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:17.541709 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16ef6b-3dd5-49f8-91cf-89cc3067234f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:17.542939 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16ef6b-3dd5-49f8-91cf-89cc3067234f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:17.543286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16ef6b-3dd5-49f8-91cf-89cc3067234f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:17.544110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16ef6b-3dd5-49f8-91cf-89cc3067234f None admin] RBAC: Adding query filter params (role.id=39be941a839d4eb6a2146c561c7a2851, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=e8a10f3332584a90a2231564527078cd) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:17.544483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16ef6b-3dd5-49f8-91cf-89cc3067234f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:17.546012 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16ef6b-3dd5-49f8-91cf-89cc3067234f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:17.557276 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 331/676] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:18:17 2022] GET /identity/v3/role_assignments?user.id=e8a10f3332584a90a2231564527078cd&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=39be941a839d4eb6a2146c561c7a2851 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:17.565186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16ef6b-3dd5-49f8-91cf-89cc3067234f None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:17.600906 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de441382-a8f3-4ab2-b4d5-3450124a4180 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:17.641975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb780649-f8b1-4c36-9fdc-3673f391c6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:17.642798 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb780649-f8b1-4c36-9fdc-3673f391c6dc None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:17.643175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb780649-f8b1-4c36-9fdc-3673f391c6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:17.643548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb780649-f8b1-4c36-9fdc-3673f391c6dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:17.655815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb780649-f8b1-4c36-9fdc-3673f391c6dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:17.657452 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb780649-f8b1-4c36-9fdc-3673f391c6dc None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:17.675658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-eb780649-f8b1-4c36-9fdc-3673f391c6dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:17.675658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:17.675658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:17.675658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:17.675658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:17.675658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:17.675658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:17.675658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:17.675658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:17.675658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:17.675658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:17.675658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:17.675658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:17.675658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:17.675658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:17.675658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:17.676889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:17.676889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:17.676889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:17.676889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:17.676889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:17.676889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:17.676889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:17.676889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:17.676889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:17.676889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:17.676889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:17.676889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:17.676889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:17.676889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:17.676889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:17.676889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:17.676889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:17.678018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:17.678018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:17.678018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:17.684398 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 346/677] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:17.684398 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb780649-f8b1-4c36-9fdc-3673f391c6dc None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:17.717183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:17.727984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16ef6b-3dd5-49f8-91cf-89cc3067234f None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:17.775097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d8edb-44e8-4936-8406-12a94c7e7888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:17.775969 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d8edb-44e8-4936-8406-12a94c7e7888 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:17.776448 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d8edb-44e8-4936-8406-12a94c7e7888 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:17.776856 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d8edb-44e8-4936-8406-12a94c7e7888 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:17.786701 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d8edb-44e8-4936-8406-12a94c7e7888 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:17.788324 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d8edb-44e8-4936-8406-12a94c7e7888 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:17.801555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-9e2d8edb-44e8-4936-8406-12a94c7e7888 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:17.801555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:17.801555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:17.801555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:17.801555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:17.801555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:17.801555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:17.801555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:17.801555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:17.801555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:17.801555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:17.801555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:17.801555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:17.801555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:17.801555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:17.802657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:17.802657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:17.802657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:17.802657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:17.802657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:17.802657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:17.802657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:17.802657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:17.802657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:17.802657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:17.802657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:17.802657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:17.802657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:17.802657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:17.802657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:17.802657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:17.802657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:17.803693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:17.803693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:17.803693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:17.803693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:17.803693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:17.803693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:17.803693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:17.803693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:17.803693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 332/678] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:17.804191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d8edb-44e8-4936-8406-12a94c7e7888 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:17.818323 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:17.828242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb780649-f8b1-4c36-9fdc-3673f391c6dc None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:17.874823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c674b-e304-4823-a72f-20fd7f52c00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:17.875733 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c674b-e304-4823-a72f-20fd7f52c00a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:17.876016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c674b-e304-4823-a72f-20fd7f52c00a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:17.876289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c674b-e304-4823-a72f-20fd7f52c00a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:17.877037 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c674b-e304-4823-a72f-20fd7f52c00a None admin] RBAC: Adding query filter params (role.id=39be941a839d4eb6a2146c561c7a2851, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=a40b1ed93a6b4e5fb0e8f907627379a7) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:17.877388 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c674b-e304-4823-a72f-20fd7f52c00a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:17.878845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c674b-e304-4823-a72f-20fd7f52c00a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:17.886614 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 347/679] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:18:17 2022] GET /identity/v3/role_assignments?user.id=a40b1ed93a6b4e5fb0e8f907627379a7&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=39be941a839d4eb6a2146c561c7a2851 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:17.887499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c674b-e304-4823-a72f-20fd7f52c00a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:17.935464 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d8edb-44e8-4936-8406-12a94c7e7888 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:17.982492 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b5a70-05c5-41b5-badc-07dfe8b0751b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:17.983556 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b5a70-05c5-41b5-badc-07dfe8b0751b None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:17.983936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b5a70-05c5-41b5-badc-07dfe8b0751b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:17.984312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b5a70-05c5-41b5-badc-07dfe8b0751b None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:17.985235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b5a70-05c5-41b5-badc-07dfe8b0751b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:17.986004 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b5a70-05c5-41b5-badc-07dfe8b0751b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:17.987999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b5a70-05c5-41b5-badc-07dfe8b0751b None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:18.011556 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c674b-e304-4823-a72f-20fd7f52c00a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:18.013977 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 333/680] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:18.019201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b5a70-05c5-41b5-badc-07dfe8b0751b None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:18.038926 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:18.066626 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d49fd-a3d4-4769-88a4-75ae7f3c7859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:18.067506 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d49fd-a3d4-4769-88a4-75ae7f3c7859 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:18.067898 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d49fd-a3d4-4769-88a4-75ae7f3c7859 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:18.068287 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d49fd-a3d4-4769-88a4-75ae7f3c7859 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:18.069121 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d49fd-a3d4-4769-88a4-75ae7f3c7859 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:18.069538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d49fd-a3d4-4769-88a4-75ae7f3c7859 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:18.071799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d49fd-a3d4-4769-88a4-75ae7f3c7859 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:18.091672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 348/681] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:18.091672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f885818-f183-4714-926e-9e9b14f6e858 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:18.091672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f885818-f183-4714-926e-9e9b14f6e858 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:18.091672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f885818-f183-4714-926e-9e9b14f6e858 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:18.091672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 349/682] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:18.096572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f885818-f183-4714-926e-9e9b14f6e858 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:18.131390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:18.156304 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b5a70-05c5-41b5-badc-07dfe8b0751b None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:18.199810 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce298e5-0ac3-4b19-9dff-25f76ab6f64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:18.200445 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce298e5-0ac3-4b19-9dff-25f76ab6f64f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:18.200724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce298e5-0ac3-4b19-9dff-25f76ab6f64f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:18.200997 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce298e5-0ac3-4b19-9dff-25f76ab6f64f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:18.211358 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce298e5-0ac3-4b19-9dff-25f76ab6f64f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:18.212914 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce298e5-0ac3-4b19-9dff-25f76ab6f64f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:18.224830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-1ce298e5-0ac3-4b19-9dff-25f76ab6f64f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 10:18:18.224830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:18.224830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:18.224830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:18.224830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:18.224830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:18.224830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:18.224830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:18.224830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:18.224830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:18.224830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:18.224830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:18.224830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:18.224830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:18.224830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:18.224830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:18.225976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:18.225976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:18.225976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:18.225976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:18.225976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:18.225976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:18.225976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:18.225976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:18.225976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:18.225976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:18.225976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:18.225976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:18.225976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:18.225976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:18.225976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:18.225976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:18.225976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:18.225976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:18.227135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:18.227135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:18.227135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:18.227135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:18.227135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:18.227135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:18.227135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:18.227135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:18.227135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 10:18:18.227135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:18.227135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 334/683] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:18:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:18:18.227893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f84c7-6f53-4a6b-a607-57ecf164ab31 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:18.227893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f84c7-6f53-4a6b-a607-57ecf164ab31 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:18.228261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f84c7-6f53-4a6b-a607-57ecf164ab31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:18.237905 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f885818-f183-4714-926e-9e9b14f6e858 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:18.248689 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-c65f84c7-6f53-4a6b-a607-57ecf164ab31 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:18.249798 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c65f84c7-6f53-4a6b-a607-57ecf164ab31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:18.293916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e044046-db8b-4ca4-bf1c-a92ba4d3d550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:18.295122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e044046-db8b-4ca4-bf1c-a92ba4d3d550 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:18.295551 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e044046-db8b-4ca4-bf1c-a92ba4d3d550 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:18.296008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e044046-db8b-4ca4-bf1c-a92ba4d3d550 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:18.296884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e044046-db8b-4ca4-bf1c-a92ba4d3d550 None admin] RBAC: Adding query filter params (role.id=39be941a839d4eb6a2146c561c7a2851, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=a282e74d177544bbbfba0fdb5e52df60) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:18.297396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e044046-db8b-4ca4-bf1c-a92ba4d3d550 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:18.298986 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e044046-db8b-4ca4-bf1c-a92ba4d3d550 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:18.305855 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 350/684] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:18:18 2022] GET /identity/v3/role_assignments?user.id=a282e74d177544bbbfba0fdb5e52df60&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=39be941a839d4eb6a2146c561c7a2851 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:18.306980 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e044046-db8b-4ca4-bf1c-a92ba4d3d550 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:18.348274 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 335/685] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:18:18.462228 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:18.475468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e044046-db8b-4ca4-bf1c-a92ba4d3d550 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:18.512625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e0c3c-76dd-4ac2-8c1b-d689b3cd838a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:18.513420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e0c3c-76dd-4ac2-8c1b-d689b3cd838a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:18.513804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e0c3c-76dd-4ac2-8c1b-d689b3cd838a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:18.514182 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e0c3c-76dd-4ac2-8c1b-d689b3cd838a None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:18.515049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e0c3c-76dd-4ac2-8c1b-d689b3cd838a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:18.515460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e0c3c-76dd-4ac2-8c1b-d689b3cd838a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:18.516989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e0c3c-76dd-4ac2-8c1b-d689b3cd838a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:18.535730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 351/686] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Sep 19 10:18:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:18.540997 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f84c7-6f53-4a6b-a607-57ecf164ab31 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:18.546973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:18.660550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f84c7-6f53-4a6b-a607-57ecf164ab31 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:18.676171 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e0c3c-76dd-4ac2-8c1b-d689b3cd838a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:18.719602 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be7dc7-fc9a-4b43-95ac-7902211f5b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:18.720216 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be7dc7-fc9a-4b43-95ac-7902211f5b83 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:18.720491 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be7dc7-fc9a-4b43-95ac-7902211f5b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:18.720761 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be7dc7-fc9a-4b43-95ac-7902211f5b83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:18.819581 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be7dc7-fc9a-4b43-95ac-7902211f5b83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:18.821080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be7dc7-fc9a-4b43-95ac-7902211f5b83 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:18.827534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e0c3c-76dd-4ac2-8c1b-d689b3cd838a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:18.830113 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:18.899522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c65479-30ef-4fe8-9b5a-829932bc85ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:18.900598 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c65479-30ef-4fe8-9b5a-829932bc85ee None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:18.901123 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c65479-30ef-4fe8-9b5a-829932bc85ee None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:18.901607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c65479-30ef-4fe8-9b5a-829932bc85ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:18.925085 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c65479-30ef-4fe8-9b5a-829932bc85ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:18.930091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-28c65479-30ef-4fe8-9b5a-829932bc85ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:18:18.930091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:18.930091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:18.930091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:18.930091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:18.930091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:18.930091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:18.930091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:18.930091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:18.930091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:18.930091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:18.930091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:18.930091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:18.930091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:18.930091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:18.930091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:18.931042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:18.931042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:18.931042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:18.931042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:18.931042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:18.931042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:18.931042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:18.931042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:18.931042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:18.931042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:18.931042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:18.931042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:18.931042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:18.931042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:18.931042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:18.931042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:18.931899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:18.931899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:18.931899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:18.931899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:18.931899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:18.931899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:18.931899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:18.931899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:18.931899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:18.931899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:18.931899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:18:18.931899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:18.941190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 352/687] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:18:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:18:18.942942 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c65479-30ef-4fe8-9b5a-829932bc85ee None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:18.958577 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:19.026309 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 336/688] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:18:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:18:19.034997 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be7dc7-fc9a-4b43-95ac-7902211f5b83 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:19.160490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c65479-30ef-4fe8-9b5a-829932bc85ee None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:19.197010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be7dc7-fc9a-4b43-95ac-7902211f5b83 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:19.204384 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:19.231909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea114c-6586-4b96-acdd-300b149ec416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:19.232877 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea114c-6586-4b96-acdd-300b149ec416 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:19.233409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea114c-6586-4b96-acdd-300b149ec416 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:19.233899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea114c-6586-4b96-acdd-300b149ec416 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:19.234766 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea114c-6586-4b96-acdd-300b149ec416 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:19.236232 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea114c-6586-4b96-acdd-300b149ec416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:19.237865 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea114c-6586-4b96-acdd-300b149ec416 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:19.255521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 353/689] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:18:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:19.261777 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea114c-6586-4b96-acdd-300b149ec416 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:19.268634 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ca48e-6540-4ee1-a5fe-dbf6026426c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:19.269595 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ca48e-6540-4ee1-a5fe-dbf6026426c6 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:19.270100 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ca48e-6540-4ee1-a5fe-dbf6026426c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:19.270593 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ca48e-6540-4ee1-a5fe-dbf6026426c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:19.283221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ca48e-6540-4ee1-a5fe-dbf6026426c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:19.285072 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ca48e-6540-4ee1-a5fe-dbf6026426c6 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:19.301083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-959ca48e-6540-4ee1-a5fe-dbf6026426c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:19.301083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:19.301083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:19.301083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:19.301083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:19.301083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:19.301083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:19.301083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:19.301083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:19.301083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:19.301083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:19.301083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:19.301083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:19.301083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:19.301083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:19.301083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:19.302067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:19.302067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:19.302067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:19.302067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:19.302067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:19.302067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:19.302067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:19.302067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:19.302067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:19.302067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:19.302067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:19.302067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:19.302067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:19.302067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:19.302067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:19.302067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:19.302067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:19.302967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:19.302967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:19.302967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:19.305035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 337/690] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:19.311219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ca48e-6540-4ee1-a5fe-dbf6026426c6 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:19.379967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:19.424230 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea114c-6586-4b96-acdd-300b149ec416 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:19.470007 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea509de-3274-4418-ba4b-f6acf5e58478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:19.471000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea509de-3274-4418-ba4b-f6acf5e58478 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:19.471488 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea509de-3274-4418-ba4b-f6acf5e58478 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:19.471978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea509de-3274-4418-ba4b-f6acf5e58478 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:19.495835 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea509de-3274-4418-ba4b-f6acf5e58478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:19.499156 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea509de-3274-4418-ba4b-f6acf5e58478 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:19.518969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-9ea509de-3274-4418-ba4b-f6acf5e58478 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:19.518969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:19.518969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:19.518969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:19.518969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:19.518969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:19.518969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:19.518969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:19.518969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:19.518969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:19.518969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:19.518969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:19.518969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:19.518969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:19.518969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:19.518969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:19.519995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:19.519995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:19.519995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:19.519995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:19.519995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:19.519995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:19.519995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:19.519995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:19.519995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:19.519995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:19.519995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:19.519995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:19.519995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:19.519995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:19.519995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:19.519995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:19.519995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:19.521004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:19.521004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:19.521004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:19.524367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 354/691] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:19.530459 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea509de-3274-4418-ba4b-f6acf5e58478 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:19.537633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ca48e-6540-4ee1-a5fe-dbf6026426c6 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:19.614753 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2bede-50a4-4053-bfde-2d50536bc5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:19.615750 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2bede-50a4-4053-bfde-2d50536bc5cf None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:19.616268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2bede-50a4-4053-bfde-2d50536bc5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:19.617404 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2bede-50a4-4053-bfde-2d50536bc5cf None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:19.619650 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2bede-50a4-4053-bfde-2d50536bc5cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:19.620713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2bede-50a4-4053-bfde-2d50536bc5cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:19.622358 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2bede-50a4-4053-bfde-2d50536bc5cf None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:19.642185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 338/692] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:19.647992 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2bede-50a4-4053-bfde-2d50536bc5cf None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:19.671321 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:19.775469 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea509de-3274-4418-ba4b-f6acf5e58478 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:19.799307 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:19.806672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2bede-50a4-4053-bfde-2d50536bc5cf None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:19.822698 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad97482-db17-46d2-b87c-eef10e29689d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:19.824273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad97482-db17-46d2-b87c-eef10e29689d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:19.824810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad97482-db17-46d2-b87c-eef10e29689d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:19.825315 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad97482-db17-46d2-b87c-eef10e29689d None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:19.826172 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad97482-db17-46d2-b87c-eef10e29689d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:19.826678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad97482-db17-46d2-b87c-eef10e29689d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:19.828269 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad97482-db17-46d2-b87c-eef10e29689d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:19.857175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 355/693] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:19.862725 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad97482-db17-46d2-b87c-eef10e29689d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:19.894122 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2b9b0-ddcd-4eaf-92c3-3ce9cef26ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:19.894772 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2b9b0-ddcd-4eaf-92c3-3ce9cef26ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:19.895056 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2b9b0-ddcd-4eaf-92c3-3ce9cef26ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:19.895329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2b9b0-ddcd-4eaf-92c3-3ce9cef26ec5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:19.908643 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2b9b0-ddcd-4eaf-92c3-3ce9cef26ec5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:19.910243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2b9b0-ddcd-4eaf-92c3-3ce9cef26ec5 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:19.922910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-bfa2b9b0-ddcd-4eaf-92c3-3ce9cef26ec5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:19.922910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:19.922910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:19.922910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:19.922910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:19.922910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:19.922910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:19.922910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:19.922910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:19.922910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:19.922910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:19.922910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:19.922910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:19.922910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:19.922910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:19.923883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:19.923883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:19.923883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:19.923883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:19.923883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:19.923883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:19.923883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:19.923883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:19.923883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:19.923883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:19.923883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:19.923883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:19.923883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:19.923883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:19.923883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:19.923883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:19.923883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:19.924774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:19.924774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:19.924774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:19.924774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:19.924774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:19.924774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:19.924774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:19.924774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:19.924774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 339/694] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:19.931604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2b9b0-ddcd-4eaf-92c3-3ce9cef26ec5 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:20.001293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad97482-db17-46d2-b87c-eef10e29689d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:20.070608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:20.090658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4293483-c641-48d1-95ac-eef33145340a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:20.091300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4293483-c641-48d1-95ac-eef33145340a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:20.093411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4293483-c641-48d1-95ac-eef33145340a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:20.093411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4293483-c641-48d1-95ac-eef33145340a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:20.095426 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2b9b0-ddcd-4eaf-92c3-3ce9cef26ec5 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:20.110266 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4293483-c641-48d1-95ac-eef33145340a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:20.112279 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4293483-c641-48d1-95ac-eef33145340a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:20.132914 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923714e5-c736-4855-8f91-e153440d3d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:20.137250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-a4293483-c641-48d1-95ac-eef33145340a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 10:18:20.137250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:20.137250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:20.137250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:20.137250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:20.137250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:20.137250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:20.137250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:20.137250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:20.137250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:20.137250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:20.137250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:20.137250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:20.137250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:20.137250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:20.137250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:20.139360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:20.139360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:20.139360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:20.139360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:20.139360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:20.139360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:20.139360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:20.139360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:20.139360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:20.139360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:20.139360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:20.139360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:20.139360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:20.139360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:20.139360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:20.139360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:20.139360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:20.139360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:20.141648 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:20.141648 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:20.141648 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:20.141648 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:20.141648 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:20.141648 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:20.141648 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:20.141648 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:20.141648 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 10:18:20.141648 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:20.142886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923714e5-c736-4855-8f91-e153440d3d1f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:20.142886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923714e5-c736-4855-8f91-e153440d3d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:20.142886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923714e5-c736-4855-8f91-e153440d3d1f None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:20.142886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923714e5-c736-4855-8f91-e153440d3d1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:20.143420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 356/695] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Sep 19 10:18:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:20.143557 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923714e5-c736-4855-8f91-e153440d3d1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:20.147320 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4293483-c641-48d1-95ac-eef33145340a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:20.147533 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923714e5-c736-4855-8f91-e153440d3d1f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:20.159418 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 340/696] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:20.165681 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923714e5-c736-4855-8f91-e153440d3d1f None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:20.238577 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:20.285659 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923714e5-c736-4855-8f91-e153440d3d1f None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:20.300411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4293483-c641-48d1-95ac-eef33145340a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:20.324134 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab7330-a05d-4f18-8dcf-3b412eeb0f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:20.324711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab7330-a05d-4f18-8dcf-3b412eeb0f27 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:20.324991 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab7330-a05d-4f18-8dcf-3b412eeb0f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:20.325299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab7330-a05d-4f18-8dcf-3b412eeb0f27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:20.326017 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab7330-a05d-4f18-8dcf-3b412eeb0f27 None admin] RBAC: Adding query filter params (role.id=39be941a839d4eb6a2146c561c7a2851, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=17a8bc6c506c45be897bda23eaab485a) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:20.326330 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab7330-a05d-4f18-8dcf-3b412eeb0f27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:20.327895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab7330-a05d-4f18-8dcf-3b412eeb0f27 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:20.331056 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:20.335534 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 341/697] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:18:20 2022] GET /identity/v3/role_assignments?user.id=17a8bc6c506c45be897bda23eaab485a&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=39be941a839d4eb6a2146c561c7a2851 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:20.337151 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af0c15-6742-4e2f-86d5-446651c61192 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:20.337433 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af0c15-6742-4e2f-86d5-446651c61192 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:20.337707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af0c15-6742-4e2f-86d5-446651c61192 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:20.343872 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 342/698] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:20.345917 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db9914-32c7-475e-b161-b5d484f5e61f None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:20.348062 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db9914-32c7-475e-b161-b5d484f5e61f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:20.348062 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db9914-32c7-475e-b161-b5d484f5e61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:20.352381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760ae01-9ad2-45fd-809d-18341fda7850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:20.353000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760ae01-9ad2-45fd-809d-18341fda7850 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:20.353596 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760ae01-9ad2-45fd-809d-18341fda7850 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:20.353862 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760ae01-9ad2-45fd-809d-18341fda7850 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:20.356134 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760ae01-9ad2-45fd-809d-18341fda7850 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:20.356459 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760ae01-9ad2-45fd-809d-18341fda7850 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:20.357957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760ae01-9ad2-45fd-809d-18341fda7850 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:20.375259 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-f8db9914-32c7-475e-b161-b5d484f5e61f None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:20.377324 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-f8db9914-32c7-475e-b161-b5d484f5e61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:20.410181 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 357/699] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Sep 19 10:18:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:20.412719 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760ae01-9ad2-45fd-809d-18341fda7850 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:20.525774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 343/700] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:18:20.528062 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac86b0-9db4-480a-9232-9b0fbfbdf700 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:20.528462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac86b0-9db4-480a-9232-9b0fbfbdf700 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:20.528739 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac86b0-9db4-480a-9232-9b0fbfbdf700 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:20.530530 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 344/701] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:20.536255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135abae2-1342-4f41-a0ef-c10865ea1d9f None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:20.536550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135abae2-1342-4f41-a0ef-c10865ea1d9f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:20.536803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135abae2-1342-4f41-a0ef-c10865ea1d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:20.572490 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-135abae2-1342-4f41-a0ef-c10865ea1d9f None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:20.574304 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-135abae2-1342-4f41-a0ef-c10865ea1d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:20.577336 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:20.582299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760ae01-9ad2-45fd-809d-18341fda7850 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:20.618227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5050d-e923-49ce-af09-0473b1c47697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:20.618817 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5050d-e923-49ce-af09-0473b1c47697 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:20.619105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5050d-e923-49ce-af09-0473b1c47697 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:20.619385 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5050d-e923-49ce-af09-0473b1c47697 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:20.652542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 345/702] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:18:20.691181 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5050d-e923-49ce-af09-0473b1c47697 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:20.692646 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5050d-e923-49ce-af09-0473b1c47697 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:20.789797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:20.794511 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 358/703] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:18:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:18:20.800023 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135abae2-1342-4f41-a0ef-c10865ea1d9f None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:20.838111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5050d-e923-49ce-af09-0473b1c47697 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:20.873448 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:20.923379 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135abae2-1342-4f41-a0ef-c10865ea1d9f None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:20.968820 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b0060-52eb-48d2-a092-08f687d4c478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:20.969524 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b0060-52eb-48d2-a092-08f687d4c478 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:20.969876 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b0060-52eb-48d2-a092-08f687d4c478 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:20.970161 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b0060-52eb-48d2-a092-08f687d4c478 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:20.982602 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b0060-52eb-48d2-a092-08f687d4c478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:20.984079 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b0060-52eb-48d2-a092-08f687d4c478 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:21.002738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-b04b0060-52eb-48d2-a092-08f687d4c478 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:21.002738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:21.002738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:21.002738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:21.002738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:21.002738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:21.002738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:21.002738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:21.002738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:21.002738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:21.002738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:21.002738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:21.002738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:21.002738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:21.002738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:21.002738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:21.003873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:21.003873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:21.003873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:21.003873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:21.003873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:21.003873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:21.003873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:21.003873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:21.003873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:21.003873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:21.003873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:21.003873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:21.003873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:21.003873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:21.003873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:21.003873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:21.003873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:21.005031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:21.005031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:21.005031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:21.005031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 346/704] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:21.005501 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78accc32-12c2-4e2f-9c0c-e9bafe3cc391 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:21.005730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78accc32-12c2-4e2f-9c0c-e9bafe3cc391 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:21.006823 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78accc32-12c2-4e2f-9c0c-e9bafe3cc391 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:21.007984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 347/705] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:21.008692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78accc32-12c2-4e2f-9c0c-e9bafe3cc391 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:21.075840 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5050d-e923-49ce-af09-0473b1c47697 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:21.111755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96693c2f-dd49-4ee7-b83b-48354e951690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:21.112383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96693c2f-dd49-4ee7-b83b-48354e951690 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:21.112659 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96693c2f-dd49-4ee7-b83b-48354e951690 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:21.112934 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96693c2f-dd49-4ee7-b83b-48354e951690 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:21.113558 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96693c2f-dd49-4ee7-b83b-48354e951690 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:21.114918 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96693c2f-dd49-4ee7-b83b-48354e951690 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:21.125576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-96693c2f-dd49-4ee7-b83b-48354e951690 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 10:18:21.125576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:21.125576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:21.125576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:21.125576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:21.125576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:21.125576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:21.125576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:21.125576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 10:18:21.125576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:21.125576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:21.125576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:21.125576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:21.125576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:21.125576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:21.125576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:21.125576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:21.126629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:21.126629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:21.126629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:21.126629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:21.126629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:21.126629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:21.126629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:21.126629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:21.126629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:21.126629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:21.126629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:21.126629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:21.126629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:21.126629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:21.126629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:21.126629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:21.130185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:21.130185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:21.130185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:21.130185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:21.130185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:21.130185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:21.130185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:21.130185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:21.130185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:21.130185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:21.130185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:21.130185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:21.130185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 10:18:21.130185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:21.130945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78accc32-12c2-4e2f-9c0c-e9bafe3cc391 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:21.131349 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 359/706] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Sep 19 10:18:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:18:21.131349 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96693c2f-dd49-4ee7-b83b-48354e951690 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:21.148939 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:21.170852 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a8d02-4b8b-4cb5-88f7-9a4343801303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:21.171897 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a8d02-4b8b-4cb5-88f7-9a4343801303 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:21.172178 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a8d02-4b8b-4cb5-88f7-9a4343801303 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:21.172455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a8d02-4b8b-4cb5-88f7-9a4343801303 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:21.184948 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a8d02-4b8b-4cb5-88f7-9a4343801303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:21.186441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a8d02-4b8b-4cb5-88f7-9a4343801303 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:21.193220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:21.201194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-3c9a8d02-4b8b-4cb5-88f7-9a4343801303 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:21.201194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:21.201194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:21.201194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:21.201194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:21.201194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:21.201194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:21.201194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:21.201194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:21.201194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:21.201194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:21.201194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:21.201194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:21.201194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:21.201194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:21.201194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:21.202424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:21.202424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:21.202424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:21.202424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:21.202424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:21.202424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:21.202424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:21.202424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:21.202424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:21.202424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:21.202424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:21.202424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:21.202424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:21.202424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:21.202424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:21.202424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:21.202424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:21.203432 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:21.203432 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:21.203432 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:21.207201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 348/707] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:21.208538 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901036e-26cc-466e-a5dc-ae5e9817ab95 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:21.208831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901036e-26cc-466e-a5dc-ae5e9817ab95 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:21.209129 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901036e-26cc-466e-a5dc-ae5e9817ab95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:21.227399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-f901036e-26cc-466e-a5dc-ae5e9817ab95 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:21.228586 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-f901036e-26cc-466e-a5dc-ae5e9817ab95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:21.235781 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96693c2f-dd49-4ee7-b83b-48354e951690 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:21.271292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5283231-1515-4f67-ac73-bf6402b80048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:21.272015 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5283231-1515-4f67-ac73-bf6402b80048 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:21.272292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5283231-1515-4f67-ac73-bf6402b80048 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:21.272561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5283231-1515-4f67-ac73-bf6402b80048 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:21.273480 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5283231-1515-4f67-ac73-bf6402b80048 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:21.273783 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5283231-1515-4f67-ac73-bf6402b80048 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:21.275312 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5283231-1515-4f67-ac73-bf6402b80048 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:21.293220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 360/708] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:21.293220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5283231-1515-4f67-ac73-bf6402b80048 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:21.306335 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 349/709] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:18:21.306335 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901036e-26cc-466e-a5dc-ae5e9817ab95 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:21.403696 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5283231-1515-4f67-ac73-bf6402b80048 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:21.427357 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:21.431189 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:21.439437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a05fa7-3e56-44c8-b85a-c1eca798bd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:21.439769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901036e-26cc-466e-a5dc-ae5e9817ab95 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:21.440434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a05fa7-3e56-44c8-b85a-c1eca798bd35 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:21.440666 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a05fa7-3e56-44c8-b85a-c1eca798bd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:21.441036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a05fa7-3e56-44c8-b85a-c1eca798bd35 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:21.441761 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a05fa7-3e56-44c8-b85a-c1eca798bd35 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:21.442139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a05fa7-3e56-44c8-b85a-c1eca798bd35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:21.443543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a05fa7-3e56-44c8-b85a-c1eca798bd35 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:21.448809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 361/710] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Sep 19 10:18:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:21.449646 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a05fa7-3e56-44c8-b85a-c1eca798bd35 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:21.484806 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ab3a5-51f8-464e-87f1-5362c840e19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:21.485460 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ab3a5-51f8-464e-87f1-5362c840e19f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:21.485736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ab3a5-51f8-464e-87f1-5362c840e19f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:21.486007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ab3a5-51f8-464e-87f1-5362c840e19f None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:21.486650 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ab3a5-51f8-464e-87f1-5362c840e19f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:21.486954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ab3a5-51f8-464e-87f1-5362c840e19f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:21.489088 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ab3a5-51f8-464e-87f1-5362c840e19f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:21.505397 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 350/711] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:21.506579 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ab3a5-51f8-464e-87f1-5362c840e19f None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:21.578018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a05fa7-3e56-44c8-b85a-c1eca798bd35 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:21.617363 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95f1ba-799a-4ae8-97a3-b3620197c8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:21.618038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95f1ba-799a-4ae8-97a3-b3620197c8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:21.618289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95f1ba-799a-4ae8-97a3-b3620197c8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:21.618564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95f1ba-799a-4ae8-97a3-b3620197c8b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:21.622866 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ab3a5-51f8-464e-87f1-5362c840e19f None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:21.625636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95f1ba-799a-4ae8-97a3-b3620197c8b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:21.627059 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95f1ba-799a-4ae8-97a3-b3620197c8b4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:21.638541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-ea95f1ba-799a-4ae8-97a3-b3620197c8b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:21.638541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:21.638541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:21.638541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:21.638541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:21.638541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:21.638541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:21.638541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:21.638541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:21.638541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:21.638541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:21.638541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:21.638541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:21.638541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:21.638541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:21.639436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:21.639436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:21.639436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:21.639436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:21.639436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:21.639436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:21.639436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:21.639436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:21.639436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:21.639436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:21.639436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:21.639436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:21.639436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:21.639436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:21.639436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:21.639436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:21.639436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:21.640337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:21.640337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:21.640337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:21.640337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:21.640337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:21.640337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:21.640337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:21.640337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:21.640337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 362/712] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:21.640337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95f1ba-799a-4ae8-97a3-b3620197c8b4 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:21.668371 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d85903-632c-43c5-84a0-1f1975d601c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:21.669001 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d85903-632c-43c5-84a0-1f1975d601c1 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:21.669348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d85903-632c-43c5-84a0-1f1975d601c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:21.669668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d85903-632c-43c5-84a0-1f1975d601c1 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:21.670265 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d85903-632c-43c5-84a0-1f1975d601c1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:21.670789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d85903-632c-43c5-84a0-1f1975d601c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:21.672019 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d85903-632c-43c5-84a0-1f1975d601c1 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:21.678327 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 351/713] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Sep 19 10:18:21 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:21.686207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d85903-632c-43c5-84a0-1f1975d601c1 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:21.692689 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:21.718277 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:21.759128 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95f1ba-799a-4ae8-97a3-b3620197c8b4 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:21.800036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb547198-43f1-4ac8-9514-8dee264f195d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:21.800739 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb547198-43f1-4ac8-9514-8dee264f195d None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:21.801105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb547198-43f1-4ac8-9514-8dee264f195d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:21.801390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb547198-43f1-4ac8-9514-8dee264f195d None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:21.802099 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb547198-43f1-4ac8-9514-8dee264f195d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:21.803559 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb547198-43f1-4ac8-9514-8dee264f195d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:21.834287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d85903-632c-43c5-84a0-1f1975d601c1 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:21.871627 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa33bb-4121-4c91-a4c8-904a1f9b03d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:21.872308 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa33bb-4121-4c91-a4c8-904a1f9b03d1 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:21.872585 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa33bb-4121-4c91-a4c8-904a1f9b03d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:21.872816 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa33bb-4121-4c91-a4c8-904a1f9b03d1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:21.873424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa33bb-4121-4c91-a4c8-904a1f9b03d1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:21.874825 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa33bb-4121-4c91-a4c8-904a1f9b03d1 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:21.880211 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 363/714] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 19 10:18:21 2022] POST /identity/v3/users => generated 254 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:18:21.881246 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb547198-43f1-4ac8-9514-8dee264f195d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:21.887597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-71aa33bb-4121-4c91-a4c8-904a1f9b03d1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 10:18:21.887597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:21.887597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:21.887597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:21.887597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:21.887597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:21.887597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:21.887597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:21.887597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 10:18:21.887597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:21.887597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:21.887597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:21.887597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:21.887597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:21.887597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:21.887597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:21.887597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:21.888729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:21.888729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:21.888729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:21.888729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:21.888729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:21.888729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:21.888729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:21.888729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:21.888729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:21.888729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:21.888729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:21.888729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:21.888729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:21.888729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:21.888729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:21.888729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:21.890064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:21.890064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:21.890064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:21.890064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:21.890064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:21.890064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:21.890064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:21.890064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:21.890064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:21.890064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:21.890064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:21.890064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:21.890064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 10:18:21.890064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:21.890064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 352/715] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:21 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:21.905919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa33bb-4121-4c91-a4c8-904a1f9b03d1 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:22.043348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb547198-43f1-4ac8-9514-8dee264f195d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:22.046853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:22.096904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:22.109452 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dac4c3-e469-42e4-9ff7-ab1b3380d422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:22.110643 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dac4c3-e469-42e4-9ff7-ab1b3380d422 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:22.111248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dac4c3-e469-42e4-9ff7-ab1b3380d422 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:22.111668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dac4c3-e469-42e4-9ff7-ab1b3380d422 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:22.112511 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dac4c3-e469-42e4-9ff7-ab1b3380d422 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:22.113128 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dac4c3-e469-42e4-9ff7-ab1b3380d422 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:22.115275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dac4c3-e469-42e4-9ff7-ab1b3380d422 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:22.121678 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa33bb-4121-4c91-a4c8-904a1f9b03d1 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:22.130579 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 364/716] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:22.139304 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dac4c3-e469-42e4-9ff7-ab1b3380d422 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:22.172796 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db7c11-dd30-465e-9414-227651368f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:22.173509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db7c11-dd30-465e-9414-227651368f8a None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:22.173788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db7c11-dd30-465e-9414-227651368f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:22.174305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db7c11-dd30-465e-9414-227651368f8a None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:22.174940 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db7c11-dd30-465e-9414-227651368f8a None admin] RBAC: Adding query filter params (name=network) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:22.175668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db7c11-dd30-465e-9414-227651368f8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:22.177353 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db7c11-dd30-465e-9414-227651368f8a None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:22.185101 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 353/717] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 19 10:18:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:22.190045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db7c11-dd30-465e-9414-227651368f8a None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:22.262286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dac4c3-e469-42e4-9ff7-ab1b3380d422 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:22.297230 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db7c11-dd30-465e-9414-227651368f8a None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:22.299745 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ca755-7521-4eef-b9d8-41d6912f4d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:22.300351 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ca755-7521-4eef-b9d8-41d6912f4d65 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:22.300632 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ca755-7521-4eef-b9d8-41d6912f4d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:22.300908 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ca755-7521-4eef-b9d8-41d6912f4d65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:22.301662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ca755-7521-4eef-b9d8-41d6912f4d65 None admin] RBAC: Adding query filter params (role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=b4007794a30247c4a318af62d05e3b92) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:22.301973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ca755-7521-4eef-b9d8-41d6912f4d65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:22.303380 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ca755-7521-4eef-b9d8-41d6912f4d65 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:22.313163 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 365/718] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:18:22 2022] GET /identity/v3/role_assignments?user.id=b4007794a30247c4a318af62d05e3b92&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:22.338981 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16417fb9-fd8c-4810-aeec-c57e43b612ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:22.339600 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16417fb9-fd8c-4810-aeec-c57e43b612ef None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:22.339885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16417fb9-fd8c-4810-aeec-c57e43b612ef None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:22.340145 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16417fb9-fd8c-4810-aeec-c57e43b612ef None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:22.340785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16417fb9-fd8c-4810-aeec-c57e43b612ef None admin] RBAC: Adding query filter params (type=network) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:22.341129 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16417fb9-fd8c-4810-aeec-c57e43b612ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:22.342649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16417fb9-fd8c-4810-aeec-c57e43b612ef None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:22.348542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 354/719] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 19 10:18:22 2022] GET /identity/v3/services?type=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:23.067503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36e84b-3673-4662-9bd8-a40d003f0c05 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:23.068514 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36e84b-3673-4662-9bd8-a40d003f0c05 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:23.069004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36e84b-3673-4662-9bd8-a40d003f0c05 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:23.070817 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 366/720] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:23.076971 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8cfcc-8e7f-4cce-9ceb-2763b9b480d4 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:23.077527 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8cfcc-8e7f-4cce-9ceb-2763b9b480d4 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:23.078018 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8cfcc-8e7f-4cce-9ceb-2763b9b480d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:23.097397 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-bfb8cfcc-8e7f-4cce-9ceb-2763b9b480d4 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:23.098927 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-bfb8cfcc-8e7f-4cce-9ceb-2763b9b480d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:23.103117 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:23.169957 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 355/721] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:18:23.518246 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a36e84b-3673-4662-9bd8-a40d003f0c05 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:23.554766 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:23.654791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a36e84b-3673-4662-9bd8-a40d003f0c05 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:23.694507 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb447e6-d61f-4239-bbb3-2881983e38a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:23.695514 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb447e6-d61f-4239-bbb3-2881983e38a9 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:23.695992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb447e6-d61f-4239-bbb3-2881983e38a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:23.696461 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb447e6-d61f-4239-bbb3-2881983e38a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:23.707549 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb447e6-d61f-4239-bbb3-2881983e38a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:23.712804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-dfb447e6-d61f-4239-bbb3-2881983e38a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:18:23.712804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:23.712804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:23.712804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:23.712804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:23.712804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:23.712804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:23.712804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:23.712804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:23.712804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:23.712804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:23.712804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:23.712804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:23.712804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:23.712804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:23.712804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:23.714338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:23.714338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:23.714338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:23.714338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:23.714338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:23.714338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:23.714338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:23.714338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:23.714338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:23.714338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:23.714338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:23.714338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:23.714338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:23.714338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:23.714338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:23.714338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:23.715225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:23.715225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:23.715225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:23.715225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:23.715225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:23.715225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:23.715225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:23.715225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:23.715225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:23.715225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:23.715225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:18:23.715225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:23.718014 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 367/722] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:18:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:23.724173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8cfcc-8e7f-4cce-9ceb-2763b9b480d4 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:23.865800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8cfcc-8e7f-4cce-9ceb-2763b9b480d4 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:23.909076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ce374-8ec5-454b-a372-bfb370f7dcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:23.910031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ce374-8ec5-454b-a372-bfb370f7dcee None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:23.910529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ce374-8ec5-454b-a372-bfb370f7dcee None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:23.911001 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ce374-8ec5-454b-a372-bfb370f7dcee None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:23.911873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ce374-8ec5-454b-a372-bfb370f7dcee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:23.912442 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ce374-8ec5-454b-a372-bfb370f7dcee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:23.914123 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ce374-8ec5-454b-a372-bfb370f7dcee None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:23.927624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 356/723] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:18:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:23.932849 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb447e6-d61f-4239-bbb3-2881983e38a9 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:24.034471 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:24.049351 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb447e6-d61f-4239-bbb3-2881983e38a9 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:24.086970 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7314027-b55e-4047-9876-15a7f2886f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:24.087564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7314027-b55e-4047-9876-15a7f2886f21 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:24.087846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7314027-b55e-4047-9876-15a7f2886f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:24.088122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7314027-b55e-4047-9876-15a7f2886f21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:24.105154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7314027-b55e-4047-9876-15a7f2886f21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:24.106591 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7314027-b55e-4047-9876-15a7f2886f21 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:24.124093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-f7314027-b55e-4047-9876-15a7f2886f21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:24.124093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:24.124093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:24.124093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:24.124093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:24.124093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:24.124093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:24.124093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:24.124093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:24.124093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:24.124093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:24.124093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:24.124093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:24.124093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:24.124093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:24.124093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:24.125193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:24.125193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:24.125193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:24.125193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:24.125193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:24.125193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:24.125193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:24.125193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:24.125193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:24.125193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:24.125193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:24.125193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:24.125193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:24.125193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:24.125193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:24.125193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:24.125193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:24.126101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:24.126101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:24.126101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:24.126101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 368/724] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:24.132287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ce374-8ec5-454b-a372-bfb370f7dcee None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:24.135933 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:24.260364 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ce374-8ec5-454b-a372-bfb370f7dcee None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:24.301900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dea6c0-dbeb-42bc-950e-71a3c6e48393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:24.302869 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dea6c0-dbeb-42bc-950e-71a3c6e48393 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:24.303370 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dea6c0-dbeb-42bc-950e-71a3c6e48393 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:24.303838 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dea6c0-dbeb-42bc-950e-71a3c6e48393 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:24.304687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dea6c0-dbeb-42bc-950e-71a3c6e48393 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:24.305212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dea6c0-dbeb-42bc-950e-71a3c6e48393 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:24.306888 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dea6c0-dbeb-42bc-950e-71a3c6e48393 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:24.330664 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 357/725] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:24.337606 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7314027-b55e-4047-9876-15a7f2886f21 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:24.444500 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b65508-f2e9-44e1-bfa2-337b2caf87b9 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:24.444826 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b65508-f2e9-44e1-bfa2-337b2caf87b9 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:24.445077 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b65508-f2e9-44e1-bfa2-337b2caf87b9 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:24.446714 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 358/726] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:24.452122 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c549d-824e-454a-9fc3-c0d231e9c47a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:24.452409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c549d-824e-454a-9fc3-c0d231e9c47a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:24.452685 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c549d-824e-454a-9fc3-c0d231e9c47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:24.471810 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-786c549d-824e-454a-9fc3-c0d231e9c47a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:24.473024 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-786c549d-824e-454a-9fc3-c0d231e9c47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:24.517746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7314027-b55e-4047-9876-15a7f2886f21 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:24.562444 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:24.565820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03116b-585f-4d1b-9f92-0e9f4238b459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:24.566692 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03116b-585f-4d1b-9f92-0e9f4238b459 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:24.567192 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03116b-585f-4d1b-9f92-0e9f4238b459 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:24.567733 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03116b-585f-4d1b-9f92-0e9f4238b459 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:24.577215 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03116b-585f-4d1b-9f92-0e9f4238b459 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:24.579505 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03116b-585f-4d1b-9f92-0e9f4238b459 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:24.585523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:24.591405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 359/727] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 10:18:24.594110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-fe03116b-585f-4d1b-9f92-0e9f4238b459 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 10:18:24.594110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:24.594110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:24.594110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:24.594110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:24.594110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:24.594110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:24.594110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:24.594110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:24.594110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:24.594110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:24.594110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:24.594110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:24.594110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:24.594110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:24.594110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:24.595042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:24.595042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:24.595042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:24.595042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:24.595042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:24.595042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:24.595042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:24.595042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:24.595042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:24.595042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:24.595042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:24.595042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:24.595042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:24.595042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:24.595042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:24.595042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:24.595042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:24.595042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:24.596057 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:24.596057 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:24.596057 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:24.596057 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:24.596057 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:24.596057 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:24.596057 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:24.596057 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:24.596057 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 10:18:24.596057 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:24.598589 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 369/728] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:18:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:24.605158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c549d-824e-454a-9fc3-c0d231e9c47a None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:24.760857 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c549d-824e-454a-9fc3-c0d231e9c47a None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:24.804667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8548f-276e-4c8a-bc84-99d06d66c5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:24.805683 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8548f-276e-4c8a-bc84-99d06d66c5b3 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:24.806197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8548f-276e-4c8a-bc84-99d06d66c5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:24.806672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8548f-276e-4c8a-bc84-99d06d66c5b3 None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:24.808118 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8548f-276e-4c8a-bc84-99d06d66c5b3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:24.808631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8548f-276e-4c8a-bc84-99d06d66c5b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:24.810277 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8548f-276e-4c8a-bc84-99d06d66c5b3 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:24.840368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 360/729] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Sep 19 10:18:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:24.846190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03116b-585f-4d1b-9f92-0e9f4238b459 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:24.934198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8548f-276e-4c8a-bc84-99d06d66c5b3 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:24.958106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03116b-585f-4d1b-9f92-0e9f4238b459 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:24.998475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:25.001152 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062add8d-7f8e-4737-a26f-56f8c36b9b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:25.001884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062add8d-7f8e-4737-a26f-56f8c36b9b38 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:25.002167 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062add8d-7f8e-4737-a26f-56f8c36b9b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:25.002438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062add8d-7f8e-4737-a26f-56f8c36b9b38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:25.065966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8548f-276e-4c8a-bc84-99d06d66c5b3 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:25.094886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:25.100466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062add8d-7f8e-4737-a26f-56f8c36b9b38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:25.102033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74aec3d-4077-4a44-9c84-9f0ae9d7e80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:25.102193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062add8d-7f8e-4737-a26f-56f8c36b9b38 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:25.102645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74aec3d-4077-4a44-9c84-9f0ae9d7e80b None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:25.102995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74aec3d-4077-4a44-9c84-9f0ae9d7e80b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:25.103267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74aec3d-4077-4a44-9c84-9f0ae9d7e80b None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:25.103998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74aec3d-4077-4a44-9c84-9f0ae9d7e80b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:25.105387 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74aec3d-4077-4a44-9c84-9f0ae9d7e80b None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:25.188994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 361/730] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 19 10:18:24 2022] POST /identity/v3/services => generated 251 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:18:25.190344 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdafaf-7755-48ee-946f-b80d2916e29e None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:25.190628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdafaf-7755-48ee-946f-b80d2916e29e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:25.190904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdafaf-7755-48ee-946f-b80d2916e29e None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:25.192433 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 362/731] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:25.198663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce89aa-5f35-41bb-a020-02b12095b568 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:25.199030 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce89aa-5f35-41bb-a020-02b12095b568 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:25.199226 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce89aa-5f35-41bb-a020-02b12095b568 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:25.205402 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 363/732] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:25 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:25.206834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a33e86-a75e-4312-9756-c0cc8f11be7d None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:25.207112 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a33e86-a75e-4312-9756-c0cc8f11be7d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:25.207384 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a33e86-a75e-4312-9756-c0cc8f11be7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:25.225993 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 370/733] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:18:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 10:18:25.228220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a975b-3eeb-4000-811f-888c27dacdaa None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:25.229292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a975b-3eeb-4000-811f-888c27dacdaa None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:25.229570 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a975b-3eeb-4000-811f-888c27dacdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:25.236390 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-b3a33e86-a75e-4312-9756-c0cc8f11be7d None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:25.237656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b3a33e86-a75e-4312-9756-c0cc8f11be7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:25.252462 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-0d0a975b-3eeb-4000-811f-888c27dacdaa None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:25.254951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0d0a975b-3eeb-4000-811f-888c27dacdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:25.325161 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 364/734] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:25 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:18:25.325995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a33e86-a75e-4312-9756-c0cc8f11be7d None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:25.348964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 371/735] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:25 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:18:25.351368 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1999a1-7a43-4884-9323-4208aa56ff67 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:25.352126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1999a1-7a43-4884-9323-4208aa56ff67 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:25.352614 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1999a1-7a43-4884-9323-4208aa56ff67 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:25.354943 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 372/736] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:25.365110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73c83b-ae53-4791-b566-27520098bd06 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:25.365110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73c83b-ae53-4791-b566-27520098bd06 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:25.365110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73c83b-ae53-4791-b566-27520098bd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:25.375570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:25.397004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7d73c83b-ae53-4791-b566-27520098bd06 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:25.398467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7d73c83b-ae53-4791-b566-27520098bd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:25.427285 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:25.477107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 373/737] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:25 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 10:18:25.521607 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a33e86-a75e-4312-9756-c0cc8f11be7d None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:25.564454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb08816-4a39-4ecd-b490-7bc7f5d4ff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:25.565340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb08816-4a39-4ecd-b490-7bc7f5d4ff97 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:25.565761 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb08816-4a39-4ecd-b490-7bc7f5d4ff97 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:25.566166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb08816-4a39-4ecd-b490-7bc7f5d4ff97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:25.577363 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb08816-4a39-4ecd-b490-7bc7f5d4ff97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:25.578991 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb08816-4a39-4ecd-b490-7bc7f5d4ff97 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:25.605854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-9bb08816-4a39-4ecd-b490-7bc7f5d4ff97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:25.605854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:25.605854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:25.605854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:25.605854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:25.605854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:25.605854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:25.605854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:25.605854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:25.605854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:25.605854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:25.605854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:25.605854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:25.605854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:25.605854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:25.605854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:25.606994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:25.606994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:25.606994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:25.606994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:25.606994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:25.606994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:25.606994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:25.606994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:25.606994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:25.606994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:25.606994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:25.606994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:25.606994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:25.606994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:25.606994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:25.606994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:25.606994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:25.608103 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:25.608103 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:25.608103 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:25.609614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 365/738] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:25.610787 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73c83b-ae53-4791-b566-27520098bd06 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:25.616737 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb08816-4a39-4ecd-b490-7bc7f5d4ff97 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:25.733054 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:25.740653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73c83b-ae53-4791-b566-27520098bd06 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:25.751672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb08816-4a39-4ecd-b490-7bc7f5d4ff97 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:25.778223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffca2cd-a0df-4e6f-85d1-b7ce97bb253c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:25.778854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffca2cd-a0df-4e6f-85d1-b7ce97bb253c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:25.779137 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffca2cd-a0df-4e6f-85d1-b7ce97bb253c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:25.779419 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffca2cd-a0df-4e6f-85d1-b7ce97bb253c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:25.790329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffca2cd-a0df-4e6f-85d1-b7ce97bb253c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:25.793646 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef29c6-5234-45b0-bde3-0017f87c5e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:25.794329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef29c6-5234-45b0-bde3-0017f87c5e48 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:25.794522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef29c6-5234-45b0-bde3-0017f87c5e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:25.794790 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef29c6-5234-45b0-bde3-0017f87c5e48 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:25.795417 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef29c6-5234-45b0-bde3-0017f87c5e48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:25.795654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-7ffca2cd-a0df-4e6f-85d1-b7ce97bb253c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:18:25.795654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:25.795654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:25.795654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:25.795654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:25.795654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:25.795654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:25.795654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:25.795654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:25.795654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:25.795654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:25.795654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:25.795654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:25.795654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:25.795654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:25.795654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:25.796545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:25.796545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:25.796545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:25.796545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:25.796545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:25.796545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:25.796545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:25.796545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:25.796545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:25.796545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:25.796545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:25.796545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:25.796545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:25.796545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:25.796545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:25.796545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:25.802086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:25.802086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:25.802086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:25.802086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:25.802086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:25.802086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:25.802086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:25.802086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:25.802086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:25.802086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:25.802086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:18:25.802086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:25.802729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef29c6-5234-45b0-bde3-0017f87c5e48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:25.802803 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 374/739] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:18:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:25.802870 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef29c6-5234-45b0-bde3-0017f87c5e48 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:25.802937 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffca2cd-a0df-4e6f-85d1-b7ce97bb253c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:25.817346 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 366/740] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:25.818418 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef29c6-5234-45b0-bde3-0017f87c5e48 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:25.825723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:25.910750 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffca2cd-a0df-4e6f-85d1-b7ce97bb253c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:25.931182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef29c6-5234-45b0-bde3-0017f87c5e48 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:25.951147 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a1153-ff6f-4228-a497-b5ca5a35f710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:25.951768 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a1153-ff6f-4228-a497-b5ca5a35f710 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:25.952042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a1153-ff6f-4228-a497-b5ca5a35f710 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:25.952311 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a1153-ff6f-4228-a497-b5ca5a35f710 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:25.962898 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a1153-ff6f-4228-a497-b5ca5a35f710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:25.967607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-d84a1153-ff6f-4228-a497-b5ca5a35f710 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:25.967607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:25.967607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:25.967607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:25.967607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:25.967607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:25.967607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:25.967607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:25.967607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:25.967607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:25.967607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:25.967607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:25.967607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:25.967607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:25.967607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:25.967607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:25.968529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:25.968529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:25.968529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:25.968529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:25.968529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:25.968529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:25.968529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:25.968529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:25.968529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:25.968529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:25.968529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:25.968529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:25.968529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:25.968529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:25.968529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:25.968529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:25.974021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:25.974021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:25.974021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:25.974021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:25.974021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:25.974021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:25.974021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:25.974021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:25.974021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:25.974021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:25.974021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:25.974021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:25.974660 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070a9ca-453a-4cfa-af69-ae5925f042c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:25.974660 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070a9ca-453a-4cfa-af69-ae5925f042c4 None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:25.974660 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070a9ca-453a-4cfa-af69-ae5925f042c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:25.974660 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070a9ca-453a-4cfa-af69-ae5925f042c4 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:25.974922 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 375/741] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:18:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:25.974988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070a9ca-453a-4cfa-af69-ae5925f042c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:25.975054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a1153-ff6f-4228-a497-b5ca5a35f710 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:25.975132 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070a9ca-453a-4cfa-af69-ae5925f042c4 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:26.039803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 367/742] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 19 10:18:25 2022] POST /identity/v3/services => generated 240 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:18:26.040985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070a9ca-453a-4cfa-af69-ae5925f042c4 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:26.055316 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:26.067353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:26.092811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a1153-ff6f-4228-a497-b5ca5a35f710 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:26.150078 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18282e80-7a43-4f62-b726-aeeab1727d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:26.151247 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18282e80-7a43-4f62-b726-aeeab1727d79 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:26.151740 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18282e80-7a43-4f62-b726-aeeab1727d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:26.152086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18282e80-7a43-4f62-b726-aeeab1727d79 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:26.152914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18282e80-7a43-4f62-b726-aeeab1727d79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:26.153343 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18282e80-7a43-4f62-b726-aeeab1727d79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:26.155201 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18282e80-7a43-4f62-b726-aeeab1727d79 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:26.171936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 376/743] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:18:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:26.171936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18282e80-7a43-4f62-b726-aeeab1727d79 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:26.175443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070a9ca-453a-4cfa-af69-ae5925f042c4 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:26.223403 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f1e48-cb95-443a-9b54-9f5a376dad13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:26.224365 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f1e48-cb95-443a-9b54-9f5a376dad13 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:26.224649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f1e48-cb95-443a-9b54-9f5a376dad13 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:26.224923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f1e48-cb95-443a-9b54-9f5a376dad13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:26.236963 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f1e48-cb95-443a-9b54-9f5a376dad13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:26.239029 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f1e48-cb95-443a-9b54-9f5a376dad13 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:26.253833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-aa3f1e48-cb95-443a-9b54-9f5a376dad13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:26.253833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:26.253833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:26.253833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:26.253833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:26.253833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:26.253833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:26.253833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:26.253833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:26.253833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:26.253833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:26.253833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:26.253833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:26.253833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:26.253833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:26.254769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:26.254769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:26.254769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:26.254769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:26.254769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:26.254769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:26.254769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:26.254769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:26.254769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:26.254769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:26.254769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:26.254769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:26.254769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:26.254769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:26.254769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:26.254769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:26.254769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:26.255742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:26.255742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:26.255742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:26.255742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:26.255742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:26.255742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:26.255742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:26.255742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:26.255742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 368/744] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:26.255742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f1e48-cb95-443a-9b54-9f5a376dad13 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:26.295296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18282e80-7a43-4f62-b726-aeeab1727d79 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:26.335521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc349379-2091-4e9d-b934-a874e4d6f5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:26.336206 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc349379-2091-4e9d-b934-a874e4d6f5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:26.336489 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc349379-2091-4e9d-b934-a874e4d6f5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:26.336766 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc349379-2091-4e9d-b934-a874e4d6f5e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:26.337942 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc349379-2091-4e9d-b934-a874e4d6f5e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:26.338778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc349379-2091-4e9d-b934-a874e4d6f5e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:26.340415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc349379-2091-4e9d-b934-a874e4d6f5e9 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:26.343130 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:26.350187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 377/745] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:18:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:26.351090 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc349379-2091-4e9d-b934-a874e4d6f5e9 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:26.361183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:26.395671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f1e48-cb95-443a-9b54-9f5a376dad13 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:26.433912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9dff45-da47-4d7e-80b6-066a860697fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:26.434659 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9dff45-da47-4d7e-80b6-066a860697fb None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:26.435060 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9dff45-da47-4d7e-80b6-066a860697fb None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:26.435435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9dff45-da47-4d7e-80b6-066a860697fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:26.446764 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9dff45-da47-4d7e-80b6-066a860697fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:26.450829 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9dff45-da47-4d7e-80b6-066a860697fb None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:26.464528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-5a9dff45-da47-4d7e-80b6-066a860697fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:26.464528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:26.464528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:26.464528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:26.464528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:26.464528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:26.464528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:26.464528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:26.464528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:26.464528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:26.464528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:26.464528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:26.464528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:26.464528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:26.464528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:26.464528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:26.465954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:26.465954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:26.465954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:26.465954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:26.465954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:26.465954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:26.465954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:26.465954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:26.465954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:26.465954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:26.465954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:26.465954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:26.465954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:26.465954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:26.465954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:26.465954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:26.465954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:26.468162 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:26.468162 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:26.468162 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:26.468162 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 369/746] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:26.468671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9dff45-da47-4d7e-80b6-066a860697fb None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:26.481559 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc349379-2091-4e9d-b934-a874e4d6f5e9 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:26.524162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461ec50-e238-4a01-8f23-36ad0961429a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:26.524774 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461ec50-e238-4a01-8f23-36ad0961429a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:26.525085 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461ec50-e238-4a01-8f23-36ad0961429a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:26.525358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461ec50-e238-4a01-8f23-36ad0961429a None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:26.526596 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461ec50-e238-4a01-8f23-36ad0961429a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:26.526904 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461ec50-e238-4a01-8f23-36ad0961429a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:26.529220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461ec50-e238-4a01-8f23-36ad0961429a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:26.544528 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 378/747] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:26.545763 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461ec50-e238-4a01-8f23-36ad0961429a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:26.581456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9dff45-da47-4d7e-80b6-066a860697fb None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:26.606325 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:26.618679 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:26.623144 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8d3af-e450-46d5-90c7-9cb6d48396e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:26.623857 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8d3af-e450-46d5-90c7-9cb6d48396e0 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:26.624227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8d3af-e450-46d5-90c7-9cb6d48396e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:26.624623 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8d3af-e450-46d5-90c7-9cb6d48396e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:26.637162 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8d3af-e450-46d5-90c7-9cb6d48396e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:26.638766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8d3af-e450-46d5-90c7-9cb6d48396e0 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:26.655789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-fbd8d3af-e450-46d5-90c7-9cb6d48396e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:26.655789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:26.655789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:26.655789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:26.655789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:26.655789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:26.655789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:26.655789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:26.655789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:26.655789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:26.655789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:26.655789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:26.655789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:26.655789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:26.655789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:26.655789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:26.656777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:26.656777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:26.656777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:26.656777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:26.656777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:26.656777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:26.656777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:26.656777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:26.656777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:26.656777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:26.656777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:26.656777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:26.656777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:26.656777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:26.656777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:26.656777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:26.656777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:26.662921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:26.662921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:26.662921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:26.662921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 370/748] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:26.662921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8d3af-e450-46d5-90c7-9cb6d48396e0 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:26.671819 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461ec50-e238-4a01-8f23-36ad0961429a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:26.712347 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e9256-91ac-42b5-8c55-ef36ce60e4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:26.713756 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e9256-91ac-42b5-8c55-ef36ce60e4ec None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:26.714178 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e9256-91ac-42b5-8c55-ef36ce60e4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:26.714611 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e9256-91ac-42b5-8c55-ef36ce60e4ec None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:26.715520 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e9256-91ac-42b5-8c55-ef36ce60e4ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:26.715976 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e9256-91ac-42b5-8c55-ef36ce60e4ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:26.717490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e9256-91ac-42b5-8c55-ef36ce60e4ec None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:26.732571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 379/749] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:26.733748 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e9256-91ac-42b5-8c55-ef36ce60e4ec None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:26.766317 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8d3af-e450-46d5-90c7-9cb6d48396e0 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:26.811560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e769fc4-0301-4cf3-9993-4c45f9ed6dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:26.812858 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e769fc4-0301-4cf3-9993-4c45f9ed6dcd None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:26.813306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e769fc4-0301-4cf3-9993-4c45f9ed6dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:26.813717 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e769fc4-0301-4cf3-9993-4c45f9ed6dcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:26.814576 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e769fc4-0301-4cf3-9993-4c45f9ed6dcd None admin] RBAC: Adding query filter params (role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=17a8bc6c506c45be897bda23eaab485a) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:26.815050 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e769fc4-0301-4cf3-9993-4c45f9ed6dcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:26.816933 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e769fc4-0301-4cf3-9993-4c45f9ed6dcd None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:26.829323 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 371/750] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:18:26 2022] GET /identity/v3/role_assignments?user.id=17a8bc6c506c45be897bda23eaab485a&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:26.830563 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e769fc4-0301-4cf3-9993-4c45f9ed6dcd None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:26.901156 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e9256-91ac-42b5-8c55-ef36ce60e4ec None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:26.915156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:26.928939 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:26.943850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fefbf95-43f2-42d4-8e22-792fb1b3f0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:26.944675 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefbf95-43f2-42d4-8e22-792fb1b3f0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:26.945078 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefbf95-43f2-42d4-8e22-792fb1b3f0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:26.945474 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefbf95-43f2-42d4-8e22-792fb1b3f0d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:26.946240 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fefbf95-43f2-42d4-8e22-792fb1b3f0d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:26.946652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fefbf95-43f2-42d4-8e22-792fb1b3f0d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:26.949761 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fefbf95-43f2-42d4-8e22-792fb1b3f0d7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:26.963756 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 380/751] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:26.972221 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fefbf95-43f2-42d4-8e22-792fb1b3f0d7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:27.068781 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e769fc4-0301-4cf3-9993-4c45f9ed6dcd None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:27.132991 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d7f35-862a-42c7-b61f-04f66bd71a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:27.137433 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d7f35-862a-42c7-b61f-04f66bd71a5b None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:27.137888 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d7f35-862a-42c7-b61f-04f66bd71a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:27.138316 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d7f35-862a-42c7-b61f-04f66bd71a5b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:27.152865 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d7f35-862a-42c7-b61f-04f66bd71a5b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:27.155387 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d7f35-862a-42c7-b61f-04f66bd71a5b None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:27.163960 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fefbf95-43f2-42d4-8e22-792fb1b3f0d7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:27.185956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-ae4d7f35-862a-42c7-b61f-04f66bd71a5b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 10:18:27.185956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:27.185956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:27.185956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:27.185956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:27.185956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:27.185956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:27.185956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:27.185956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:27.185956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:27.185956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:27.185956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:27.185956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:27.185956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:27.185956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:27.185956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:27.187175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:27.187175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:27.187175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:27.187175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:27.187175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:27.187175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:27.187175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:27.187175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:27.187175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:27.187175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:27.187175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:27.187175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:27.187175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:27.187175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:27.187175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:27.187175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:27.187175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:27.187175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:27.188521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:27.188521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:27.188521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:27.188521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:27.188521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:27.188521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:27.188521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:27.188521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:27.188521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 10:18:27.188521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:27.188521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 372/752] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Sep 19 10:18:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:27.192881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d7f35-862a-42c7-b61f-04f66bd71a5b None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:27.220063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c3ef9-f20e-45d6-8f6c-770668dcb279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:27.220894 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c3ef9-f20e-45d6-8f6c-770668dcb279 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:27.226168 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c3ef9-f20e-45d6-8f6c-770668dcb279 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:27.226168 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c3ef9-f20e-45d6-8f6c-770668dcb279 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:27.237334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c3ef9-f20e-45d6-8f6c-770668dcb279 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:27.237764 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c3ef9-f20e-45d6-8f6c-770668dcb279 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:27.256890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-437c3ef9-f20e-45d6-8f6c-770668dcb279 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 10:18:27.256890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:27.256890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:27.256890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:27.256890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:27.256890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:27.256890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:27.256890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:27.256890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:27.256890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:27.256890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:27.256890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:27.256890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:27.256890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:27.256890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:27.256890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:27.257924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:27.257924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:27.257924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:27.257924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:27.257924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:27.257924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:27.257924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:27.257924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:27.257924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:27.257924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:27.257924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:27.257924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:27.257924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:27.257924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:27.257924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:27.257924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:27.257924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:27.257924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:27.258872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:27.258872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:27.258872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:27.258872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:27.258872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:27.258872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:27.258872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:27.258872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:27.258872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 10:18:27.258872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:27.259679 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 381/753] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 19 10:18:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:27.268613 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c3ef9-f20e-45d6-8f6c-770668dcb279 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:27.335572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:27.387486 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:27.397612 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d7f35-862a-42c7-b61f-04f66bd71a5b None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:27.457954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c3ef9-f20e-45d6-8f6c-770668dcb279 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:27.471601 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95d024-4d4e-4f1e-9d7a-4df4e2b3e468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:27.472170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95d024-4d4e-4f1e-9d7a-4df4e2b3e468 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:27.472454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95d024-4d4e-4f1e-9d7a-4df4e2b3e468 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:27.472733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95d024-4d4e-4f1e-9d7a-4df4e2b3e468 None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:27.473486 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95d024-4d4e-4f1e-9d7a-4df4e2b3e468 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:27.473794 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95d024-4d4e-4f1e-9d7a-4df4e2b3e468 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:27.475520 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95d024-4d4e-4f1e-9d7a-4df4e2b3e468 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:27.496351 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 373/754] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Sep 19 10:18:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:27.501427 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95d024-4d4e-4f1e-9d7a-4df4e2b3e468 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:27.528183 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d04bfa-98c8-42d6-bd85-ff0c075f9eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:27.528808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d04bfa-98c8-42d6-bd85-ff0c075f9eda None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:27.529105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d04bfa-98c8-42d6-bd85-ff0c075f9eda None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:27.529383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d04bfa-98c8-42d6-bd85-ff0c075f9eda None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:27.530379 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d04bfa-98c8-42d6-bd85-ff0c075f9eda None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:27.531697 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d04bfa-98c8-42d6-bd85-ff0c075f9eda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:27.533206 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d04bfa-98c8-42d6-bd85-ff0c075f9eda None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:27.570552 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 382/755] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Sep 19 10:18:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:27.576712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d04bfa-98c8-42d6-bd85-ff0c075f9eda None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:27.672692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95d024-4d4e-4f1e-9d7a-4df4e2b3e468 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:27.713875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb88784-6d07-4a28-a5be-74d43d86305f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:27.714658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb88784-6d07-4a28-a5be-74d43d86305f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:27.715070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb88784-6d07-4a28-a5be-74d43d86305f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:27.715542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb88784-6d07-4a28-a5be-74d43d86305f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:27.727464 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb88784-6d07-4a28-a5be-74d43d86305f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:27.729276 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb88784-6d07-4a28-a5be-74d43d86305f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:27.738949 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:27.746270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d04bfa-98c8-42d6-bd85-ff0c075f9eda None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:27.746535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-ebb88784-6d07-4a28-a5be-74d43d86305f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:27.746535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:27.746535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:27.746535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:27.746535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:27.746535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:27.746535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:27.746535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:27.746535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:27.746535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:27.746535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:27.746535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:27.746535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:27.746535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:27.746535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:27.746535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:27.747718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:27.747718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:27.747718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:27.747718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:27.747718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:27.747718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:27.747718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:27.747718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:27.747718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:27.747718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:27.747718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:27.747718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:27.747718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:27.747718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:27.747718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:27.747718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:27.747718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:27.748828 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:27.748828 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:27.748828 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:27.750194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 374/756] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:27.752267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b2e50-0c82-4380-868e-f358686f7141 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:27.752267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b2e50-0c82-4380-868e-f358686f7141 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:27.752267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b2e50-0c82-4380-868e-f358686f7141 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:27.757813 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 375/757] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:27.764522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b2e50-0c82-4380-868e-f358686f7141 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:27.769856 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:27.792111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c91ab7-ffc8-4d3b-8b6e-528b5041fa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:27.792996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c91ab7-ffc8-4d3b-8b6e-528b5041fa82 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:27.795875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c91ab7-ffc8-4d3b-8b6e-528b5041fa82 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:27.795875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c91ab7-ffc8-4d3b-8b6e-528b5041fa82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:27.872377 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c91ab7-ffc8-4d3b-8b6e-528b5041fa82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:27.873943 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c91ab7-ffc8-4d3b-8b6e-528b5041fa82 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:27.881320 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b2e50-0c82-4380-868e-f358686f7141 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:27.921633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0308a1d-f8cf-4cd9-9c86-e0159e5a78d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:27.922500 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0308a1d-f8cf-4cd9-9c86-e0159e5a78d6 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:27.922901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0308a1d-f8cf-4cd9-9c86-e0159e5a78d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:27.923277 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0308a1d-f8cf-4cd9-9c86-e0159e5a78d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:27.924106 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0308a1d-f8cf-4cd9-9c86-e0159e5a78d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:27.924512 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0308a1d-f8cf-4cd9-9c86-e0159e5a78d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:27.926228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0308a1d-f8cf-4cd9-9c86-e0159e5a78d6 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:27.940485 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 376/758] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:27.941892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09174e-183a-4857-b721-4b07d29637f5 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:27.942275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09174e-183a-4857-b721-4b07d29637f5 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:27.942682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09174e-183a-4857-b721-4b07d29637f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:27.965861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-4e09174e-183a-4857-b721-4b07d29637f5 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:27.967394 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4e09174e-183a-4857-b721-4b07d29637f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:27.986753 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:27.993651 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 383/759] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:18:27 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:18:27.995215 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c91ab7-ffc8-4d3b-8b6e-528b5041fa82 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:28.047265 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:28.065087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 377/760] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:28.065087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09174e-183a-4857-b721-4b07d29637f5 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:28.154155 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c91ab7-ffc8-4d3b-8b6e-528b5041fa82 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:28.230360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805140e-7316-4405-8e4a-c408dc735d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:28.231015 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805140e-7316-4405-8e4a-c408dc735d13 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:28.231258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805140e-7316-4405-8e4a-c408dc735d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:28.231529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805140e-7316-4405-8e4a-c408dc735d13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:28.243536 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09174e-183a-4857-b721-4b07d29637f5 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:28.254187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805140e-7316-4405-8e4a-c408dc735d13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:28.255653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805140e-7316-4405-8e4a-c408dc735d13 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:28.285118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-d805140e-7316-4405-8e4a-c408dc735d13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:28.285118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:28.285118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:28.285118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:28.285118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:28.285118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:28.285118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:28.285118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:28.285118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:28.285118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:28.285118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:28.285118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:28.285118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:28.285118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:28.285118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:28.286084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:28.286084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:28.286084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:28.286084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:28.286084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:28.286084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:28.286084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:28.286084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:28.286084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:28.286084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:28.286084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:28.286084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:28.286084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:28.286084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:28.286084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:28.286084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:28.286084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:28.286994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:28.286994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:28.286994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:28.286994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:28.286994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:28.286994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:28.286994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:28.286994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:28.286994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 384/761] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:28.290607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805140e-7316-4405-8e4a-c408dc735d13 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:28.297249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329e99d-3b5f-4563-88fd-6503d6ce4317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:28.298278 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329e99d-3b5f-4563-88fd-6503d6ce4317 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:28.298817 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329e99d-3b5f-4563-88fd-6503d6ce4317 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:28.299323 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329e99d-3b5f-4563-88fd-6503d6ce4317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:28.318552 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329e99d-3b5f-4563-88fd-6503d6ce4317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:28.319732 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329e99d-3b5f-4563-88fd-6503d6ce4317 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:28.336456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-2329e99d-3b5f-4563-88fd-6503d6ce4317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:28.336456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:28.336456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:28.336456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:28.336456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:28.336456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:28.336456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:28.336456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:28.336456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:28.336456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:28.336456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:28.336456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:28.336456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:28.336456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:28.336456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:28.336456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:28.337509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:28.337509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:28.337509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:28.337509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:28.337509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:28.337509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:28.337509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:28.337509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:28.337509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:28.337509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:28.337509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:28.337509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:28.337509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:28.337509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:28.337509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:28.337509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:28.337509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:28.338958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:28.338958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:28.338958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:28.338958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 378/762] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:28.345083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329e99d-3b5f-4563-88fd-6503d6ce4317 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:28.382123 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:28.403952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:28.435029 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805140e-7316-4405-8e4a-c408dc735d13 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:28.461977 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329e99d-3b5f-4563-88fd-6503d6ce4317 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:28.478362 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3279b14-5bf5-41c3-a4bb-b08c9bee167d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:28.479577 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3279b14-5bf5-41c3-a4bb-b08c9bee167d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:28.479859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3279b14-5bf5-41c3-a4bb-b08c9bee167d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:28.480135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3279b14-5bf5-41c3-a4bb-b08c9bee167d None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:28.480818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3279b14-5bf5-41c3-a4bb-b08c9bee167d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:28.481149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3279b14-5bf5-41c3-a4bb-b08c9bee167d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:28.482575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3279b14-5bf5-41c3-a4bb-b08c9bee167d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:28.498380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50c788-bb84-4a17-a002-f75ad5b96c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:28.498920 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50c788-bb84-4a17-a002-f75ad5b96c71 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:28.499196 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50c788-bb84-4a17-a002-f75ad5b96c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:28.499467 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50c788-bb84-4a17-a002-f75ad5b96c71 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:28.500092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50c788-bb84-4a17-a002-f75ad5b96c71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:28.500390 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50c788-bb84-4a17-a002-f75ad5b96c71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:28.501409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 385/763] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:28.501939 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50c788-bb84-4a17-a002-f75ad5b96c71 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:28.502283 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3279b14-5bf5-41c3-a4bb-b08c9bee167d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:28.518198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 379/764] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:28.518198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0e8fb-2e34-4b82-b90b-d1aa29fbff11 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:28.518198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0e8fb-2e34-4b82-b90b-d1aa29fbff11 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:28.518198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0e8fb-2e34-4b82-b90b-d1aa29fbff11 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:28.518198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 380/765] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:28.518198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0e8fb-2e34-4b82-b90b-d1aa29fbff11 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:28.615114 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3279b14-5bf5-41c3-a4bb-b08c9bee167d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:28.625728 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0e8fb-2e34-4b82-b90b-d1aa29fbff11 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:28.635542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:28.636509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:28.655051 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd5de2-63f0-423a-a8ba-3dba9e57f5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:28.655680 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd5de2-63f0-423a-a8ba-3dba9e57f5a2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:28.655962 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd5de2-63f0-423a-a8ba-3dba9e57f5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:28.656297 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd5de2-63f0-423a-a8ba-3dba9e57f5a2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:28.656936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd5de2-63f0-423a-a8ba-3dba9e57f5a2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:28.658351 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd5de2-63f0-423a-a8ba-3dba9e57f5a2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:28.662227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67a857-561e-407b-b084-35fc44bdb9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:28.662888 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67a857-561e-407b-b084-35fc44bdb9d0 None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:28.663172 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67a857-561e-407b-b084-35fc44bdb9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:28.663447 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67a857-561e-407b-b084-35fc44bdb9d0 None admin] PATH_INFO: `/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/b4007794a30247c4a318af62d05e3b92/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:28.665357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67a857-561e-407b-b084-35fc44bdb9d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e45b8fbe79994e7585095328a7ec9260, user_id=b4007794a30247c4a318af62d05e3b92, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:28.667322 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67a857-561e-407b-b084-35fc44bdb9d0 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:28.670638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-e3bd5de2-63f0-423a-a8ba-3dba9e57f5a2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 10:18:28.670638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:28.670638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:28.670638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:28.670638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:28.670638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:28.670638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:28.670638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:28.670638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 10:18:28.670638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:28.670638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:28.670638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:28.670638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:28.670638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:28.670638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:28.670638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:28.670638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:28.671886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:28.671886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:28.671886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:28.671886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:28.671886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:28.671886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:28.671886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:28.671886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:28.671886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:28.671886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:28.671886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:28.671886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:28.671886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:28.671886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:28.671886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:28.671886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:28.672775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:28.672775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:28.672775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:28.672775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:28.672775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:28.672775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:28.672775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:28.672775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:28.672775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:28.672775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:28.672775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:28.672775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:28.672775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 10:18:28.672775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:28.672775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 386/766] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Sep 19 10:18:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:28.673709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd5de2-63f0-423a-a8ba-3dba9e57f5a2 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:28.753479 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 381/767] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:18:28 2022] PUT /identity/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/b4007794a30247c4a318af62d05e3b92/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:18:28.755060 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c4ee4-9d75-435d-8fdf-cec620b9f8c7 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:28.755554 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c4ee4-9d75-435d-8fdf-cec620b9f8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:28.755848 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c4ee4-9d75-435d-8fdf-cec620b9f8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:28.779194 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd5de2-63f0-423a-a8ba-3dba9e57f5a2 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:28.789747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-e36c4ee4-9d75-435d-8fdf-cec620b9f8c7 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:28.791989 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e36c4ee4-9d75-435d-8fdf-cec620b9f8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:28.836891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeab09e-dfc8-41c7-a741-0e1209e29b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:28.837694 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeab09e-dfc8-41c7-a741-0e1209e29b8d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:28.838879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeab09e-dfc8-41c7-a741-0e1209e29b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:28.839228 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeab09e-dfc8-41c7-a741-0e1209e29b8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:28.849566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeab09e-dfc8-41c7-a741-0e1209e29b8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:28.851244 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeab09e-dfc8-41c7-a741-0e1209e29b8d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:28.867572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-adeab09e-dfc8-41c7-a741-0e1209e29b8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:28.867572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:28.867572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:28.867572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:28.867572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:28.867572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:28.867572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:28.867572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:28.867572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:28.867572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:28.867572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:28.867572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:28.867572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:28.867572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:28.867572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:28.868538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:28.868538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:28.868538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:28.868538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:28.868538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:28.868538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:28.868538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:28.868538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:28.868538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:28.868538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:28.868538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:28.868538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:28.868538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:28.868538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:28.868538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:28.868538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:28.868538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:28.870116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:28.870116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:28.870116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:28.870116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:28.870116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:28.870116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:28.870116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:28.870116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:28.873571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 387/768] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:28.874796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeab09e-dfc8-41c7-a741-0e1209e29b8d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:28.894009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 382/769] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:28.894763 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c4ee4-9d75-435d-8fdf-cec620b9f8c7 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:28.940700 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:29.020196 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeab09e-dfc8-41c7-a741-0e1209e29b8d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:29.029459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:29.035505 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c4ee4-9d75-435d-8fdf-cec620b9f8c7 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:29.059686 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c104ffc-8166-45d8-bd2e-67d84998856d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:29.060581 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c104ffc-8166-45d8-bd2e-67d84998856d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:29.061033 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c104ffc-8166-45d8-bd2e-67d84998856d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:29.062388 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c104ffc-8166-45d8-bd2e-67d84998856d None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:29.063164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c104ffc-8166-45d8-bd2e-67d84998856d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:29.063650 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c104ffc-8166-45d8-bd2e-67d84998856d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:29.065275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c104ffc-8166-45d8-bd2e-67d84998856d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:29.071590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 388/770] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Sep 19 10:18:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:29.078435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef1dfa-321b-48c1-ac8a-0b7cd4657408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:29.078943 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c104ffc-8166-45d8-bd2e-67d84998856d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:29.079434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef1dfa-321b-48c1-ac8a-0b7cd4657408 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:29.079849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef1dfa-321b-48c1-ac8a-0b7cd4657408 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:29.080286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef1dfa-321b-48c1-ac8a-0b7cd4657408 None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:29.081248 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef1dfa-321b-48c1-ac8a-0b7cd4657408 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:29.081722 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef1dfa-321b-48c1-ac8a-0b7cd4657408 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:29.085008 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef1dfa-321b-48c1-ac8a-0b7cd4657408 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:29.108935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 383/771] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:29.115453 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef1dfa-321b-48c1-ac8a-0b7cd4657408 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:29.248576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c104ffc-8166-45d8-bd2e-67d84998856d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:29.250971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:29.252177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef1dfa-321b-48c1-ac8a-0b7cd4657408 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:29.283964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824035dd-4cb6-4432-bc1c-e72c40d37cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:29.284496 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824035dd-4cb6-4432-bc1c-e72c40d37cbe None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:29.284832 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824035dd-4cb6-4432-bc1c-e72c40d37cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:29.285052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824035dd-4cb6-4432-bc1c-e72c40d37cbe None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:29.285752 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824035dd-4cb6-4432-bc1c-e72c40d37cbe None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:29.286012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824035dd-4cb6-4432-bc1c-e72c40d37cbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:29.287362 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e3710-7bc9-4c3d-883b-f73a3d5b6f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:29.287495 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824035dd-4cb6-4432-bc1c-e72c40d37cbe None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:29.287942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e3710-7bc9-4c3d-883b-f73a3d5b6f8a None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:29.288220 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e3710-7bc9-4c3d-883b-f73a3d5b6f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:29.288489 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e3710-7bc9-4c3d-883b-f73a3d5b6f8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:29.289227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e3710-7bc9-4c3d-883b-f73a3d5b6f8a None admin] RBAC: Adding query filter params (role.id=39be941a839d4eb6a2146c561c7a2851, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=4ae237601b4a4aab9e969fedb5ee37d7) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:29.289533 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e3710-7bc9-4c3d-883b-f73a3d5b6f8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:29.290983 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e3710-7bc9-4c3d-883b-f73a3d5b6f8a None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:29.302557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 389/772] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Sep 19 10:18:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:29.302557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824035dd-4cb6-4432-bc1c-e72c40d37cbe None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:29.302807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 384/773] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:18:29 2022] GET /identity/v3/role_assignments?user.id=4ae237601b4a4aab9e969fedb5ee37d7&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=39be941a839d4eb6a2146c561c7a2851 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:29.302807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037a705-91c1-4702-9940-ce9a8acd4e16 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:29.302807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037a705-91c1-4702-9940-ce9a8acd4e16 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:29.302807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037a705-91c1-4702-9940-ce9a8acd4e16 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:29.302807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 385/774] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:29.302807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037a705-91c1-4702-9940-ce9a8acd4e16 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:29.333650 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:29.432501 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037a705-91c1-4702-9940-ce9a8acd4e16 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:29.443960 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824035dd-4cb6-4432-bc1c-e72c40d37cbe None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:29.473987 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7149d5-3290-4a43-ad3a-501b36e8eb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:29.474666 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7149d5-3290-4a43-ad3a-501b36e8eb5b None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:29.474944 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7149d5-3290-4a43-ad3a-501b36e8eb5b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:29.475216 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7149d5-3290-4a43-ad3a-501b36e8eb5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:29.476064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7149d5-3290-4a43-ad3a-501b36e8eb5b None admin] RBAC: Adding query filter params (interface=public, service_id=e32b626abecb4b18aa228f3dcaf4dcf7, region_id=RegionOne) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:29.476373 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7149d5-3290-4a43-ad3a-501b36e8eb5b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:29.478117 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7149d5-3290-4a43-ad3a-501b36e8eb5b None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:29.487583 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 386/775] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Sep 19 10:18:29 2022] GET /identity/v3/endpoints?service_id=e32b626abecb4b18aa228f3dcaf4dcf7&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:29.488966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbf426-8b06-4c13-8da5-0d96decc9db5 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:29.489294 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbf426-8b06-4c13-8da5-0d96decc9db5 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:29.489613 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbf426-8b06-4c13-8da5-0d96decc9db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:29.490478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9bc23-dd17-4da5-b4fd-e0ee6a54aa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:29.491185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9bc23-dd17-4da5-b4fd-e0ee6a54aa4a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:29.491561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9bc23-dd17-4da5-b4fd-e0ee6a54aa4a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:29.491911 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9bc23-dd17-4da5-b4fd-e0ee6a54aa4a None admin] PATH_INFO: `/v3/services/network` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:29.492639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9bc23-dd17-4da5-b4fd-e0ee6a54aa4a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:29.494080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9bc23-dd17-4da5-b4fd-e0ee6a54aa4a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:29.504093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-1fe9bc23-dd17-4da5-b4fd-e0ee6a54aa4a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 10:18:29.504093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:29.504093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:29.504093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:29.504093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:29.504093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:29.504093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:29.504093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:29.504093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 10:18:29.504093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:29.504093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:29.504093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:29.504093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:29.504093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:29.504093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:29.504093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:29.504093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:29.505169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:29.505169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:29.505169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:29.505169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:29.505169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:29.505169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:29.505169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:29.505169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:29.505169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:29.505169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:29.505169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:29.505169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:29.505169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:29.505169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:29.505169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:29.505169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:29.506012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:29.506012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:29.506012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:29.506012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:29.506012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:29.506012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:29.506012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:29.506012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:29.506012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:29.506012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:29.506012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:29.506012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:29.506012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 10:18:29.506012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:29.509347 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 390/776] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:29 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:29.511348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-72cbf426-8b06-4c13-8da5-0d96decc9db5 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:29.512533 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-72cbf426-8b06-4c13-8da5-0d96decc9db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:29.516720 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9bc23-dd17-4da5-b4fd-e0ee6a54aa4a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:29.547466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:29.598502 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 387/777] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:29.599469 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbf426-8b06-4c13-8da5-0d96decc9db5 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:29.625438 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:29.635809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9bc23-dd17-4da5-b4fd-e0ee6a54aa4a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:29.680866 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f03a7-9d3b-4c8c-92c6-c59fad215246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:29.682527 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f03a7-9d3b-4c8c-92c6-c59fad215246 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:29.682950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f03a7-9d3b-4c8c-92c6-c59fad215246 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:29.683403 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f03a7-9d3b-4c8c-92c6-c59fad215246 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:29.684286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f03a7-9d3b-4c8c-92c6-c59fad215246 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:29.685958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f03a7-9d3b-4c8c-92c6-c59fad215246 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:29.691955 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 391/778] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Sep 19 10:18:29 2022] GET /identity/v3/services => generated 772 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:29.741480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbf426-8b06-4c13-8da5-0d96decc9db5 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:29.800423 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a767af6-6690-4f9f-8f3d-ee3d7c691dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:29.801389 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a767af6-6690-4f9f-8f3d-ee3d7c691dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:29.801861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a767af6-6690-4f9f-8f3d-ee3d7c691dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:29.802306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a767af6-6690-4f9f-8f3d-ee3d7c691dc9 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:29.803138 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a767af6-6690-4f9f-8f3d-ee3d7c691dc9 None admin] RBAC: Adding query filter params (name=network) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:29.803640 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a767af6-6690-4f9f-8f3d-ee3d7c691dc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:29.805363 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a767af6-6690-4f9f-8f3d-ee3d7c691dc9 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:29.817086 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 388/779] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 19 10:18:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:29.823219 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f03a7-9d3b-4c8c-92c6-c59fad215246 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:29.927712 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a767af6-6690-4f9f-8f3d-ee3d7c691dc9 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:29.956305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:29.963269 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f03a7-9d3b-4c8c-92c6-c59fad215246 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:29.999582 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88b754-5f50-420d-8aa9-205ad50c0cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:30.000209 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88b754-5f50-420d-8aa9-205ad50c0cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:30.000490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88b754-5f50-420d-8aa9-205ad50c0cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:30.000764 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88b754-5f50-420d-8aa9-205ad50c0cd6 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:30.001440 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88b754-5f50-420d-8aa9-205ad50c0cd6 None admin] RBAC: Adding query filter params (type=network) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:30.001750 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88b754-5f50-420d-8aa9-205ad50c0cd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:30.003210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88b754-5f50-420d-8aa9-205ad50c0cd6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:30.009275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 392/780] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 19 10:18:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:30.015398 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88b754-5f50-420d-8aa9-205ad50c0cd6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:30.140525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88b754-5f50-420d-8aa9-205ad50c0cd6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:30.182677 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27298677-10d1-45cd-832d-20b33b4dde3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:30.183294 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27298677-10d1-45cd-832d-20b33b4dde3e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:30.183574 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27298677-10d1-45cd-832d-20b33b4dde3e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:30.183907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27298677-10d1-45cd-832d-20b33b4dde3e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:30.185222 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27298677-10d1-45cd-832d-20b33b4dde3e None admin] RBAC: Adding query filter params (interface=public, service_id=4fb718ed0a584e67b77d3d4fb78b141b, region_id=RegionOne) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:30.185539 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27298677-10d1-45cd-832d-20b33b4dde3e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:30.186917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27298677-10d1-45cd-832d-20b33b4dde3e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:30.195807 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 393/781] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Sep 19 10:18:30 2022] GET /identity/v3/endpoints?service_id=4fb718ed0a584e67b77d3d4fb78b141b&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:30.201967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27298677-10d1-45cd-832d-20b33b4dde3e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:30.202797 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:30.231727 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:30.251096 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a767af6-6690-4f9f-8f3d-ee3d7c691dc9 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:30.318894 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27298677-10d1-45cd-832d-20b33b4dde3e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:30.328944 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2b884-c5fa-4f6e-8fc6-77a056567fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:30.329617 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2b884-c5fa-4f6e-8fc6-77a056567fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:30.329899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2b884-c5fa-4f6e-8fc6-77a056567fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:30.330169 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2b884-c5fa-4f6e-8fc6-77a056567fa8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:30.356128 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12f622-53b4-4805-9ce5-9895079613b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:30.357248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12f622-53b4-4805-9ce5-9895079613b6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:30.357739 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12f622-53b4-4805-9ce5-9895079613b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:30.358195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12f622-53b4-4805-9ce5-9895079613b6 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:30.359036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12f622-53b4-4805-9ce5-9895079613b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:30.360607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12f622-53b4-4805-9ce5-9895079613b6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:30.365669 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2b884-c5fa-4f6e-8fc6-77a056567fa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:30.367013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 394/782] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Sep 19 10:18:30 2022] GET /identity/v3/services => generated 772 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:30.371269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-dfb2b884-c5fa-4f6e-8fc6-77a056567fa8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:18:30.371269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:30.371269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:30.371269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:30.371269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:30.371269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:30.371269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:30.371269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:30.371269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:30.371269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:30.371269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:30.371269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:30.371269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:30.371269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:30.371269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:30.371269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:30.372235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:30.372235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:30.372235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:30.372235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:30.372235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:30.372235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:30.372235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:30.372235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:30.372235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:30.372235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:30.372235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:30.372235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:30.372235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:30.372235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:30.372235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:30.372235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:30.373131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:30.373131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:30.373131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:30.373131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:30.373131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:30.373131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:30.373131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:30.373131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:30.373131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:30.373131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:30.373131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:18:30.373131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:30.373131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 389/783] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:18:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 446 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:30.379381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12f622-53b4-4805-9ce5-9895079613b6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:30.500458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:30.514577 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12f622-53b4-4805-9ce5-9895079613b6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:30.559548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd8208-410b-441f-a2b8-fe372bc4b6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:30.560515 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd8208-410b-441f-a2b8-fe372bc4b6e5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:30.560978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd8208-410b-441f-a2b8-fe372bc4b6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:30.561454 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd8208-410b-441f-a2b8-fe372bc4b6e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:30.562335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd8208-410b-441f-a2b8-fe372bc4b6e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:30.562821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd8208-410b-441f-a2b8-fe372bc4b6e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:30.564353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd8208-410b-441f-a2b8-fe372bc4b6e5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:30.577310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 395/784] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:18:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:30.583173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2b884-c5fa-4f6e-8fc6-77a056567fa8 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:30.713443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2b884-c5fa-4f6e-8fc6-77a056567fa8 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:30.751288 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df0ccc-3f9e-4c64-9326-a65687fe1915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:30.752207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df0ccc-3f9e-4c64-9326-a65687fe1915 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:30.752681 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df0ccc-3f9e-4c64-9326-a65687fe1915 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:30.753161 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df0ccc-3f9e-4c64-9326-a65687fe1915 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:30.763190 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:30.767333 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df0ccc-3f9e-4c64-9326-a65687fe1915 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:30.768979 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df0ccc-3f9e-4c64-9326-a65687fe1915 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:30.784007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-27df0ccc-3f9e-4c64-9326-a65687fe1915 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:30.784007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:30.784007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:30.784007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:30.784007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:30.784007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:30.784007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:30.784007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:30.784007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:30.784007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:30.784007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:30.784007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:30.784007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:30.784007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:30.784007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:30.784007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:30.784995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:30.784995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:30.784995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:30.784995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:30.784995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:30.784995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:30.784995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:30.784995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:30.784995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:30.784995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:30.784995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:30.784995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:30.784995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:30.784995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:30.784995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:30.784995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:30.784995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:30.786065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:30.786065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:30.786065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:30.787787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 390/785] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:30.793259 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd8208-410b-441f-a2b8-fe372bc4b6e5 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:30.913837 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd8208-410b-441f-a2b8-fe372bc4b6e5 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:30.949370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579e1c4-e744-4d65-bee0-a51a6c7be8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:30.950113 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579e1c4-e744-4d65-bee0-a51a6c7be8dc None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:30.950405 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579e1c4-e744-4d65-bee0-a51a6c7be8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:30.950684 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579e1c4-e744-4d65-bee0-a51a6c7be8dc None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:30.951424 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579e1c4-e744-4d65-bee0-a51a6c7be8dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:30.951733 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579e1c4-e744-4d65-bee0-a51a6c7be8dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:30.953121 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579e1c4-e744-4d65-bee0-a51a6c7be8dc None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:30.966389 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 396/786] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:30.971985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df0ccc-3f9e-4c64-9326-a65687fe1915 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:31.124771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df0ccc-3f9e-4c64-9326-a65687fe1915 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:31.167576 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746be1d-b70b-4c7f-9f3f-414a3e40020f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:31.168213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746be1d-b70b-4c7f-9f3f-414a3e40020f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:31.168493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746be1d-b70b-4c7f-9f3f-414a3e40020f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:31.168771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746be1d-b70b-4c7f-9f3f-414a3e40020f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:31.176236 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746be1d-b70b-4c7f-9f3f-414a3e40020f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:31.177703 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746be1d-b70b-4c7f-9f3f-414a3e40020f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:31.189191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-2746be1d-b70b-4c7f-9f3f-414a3e40020f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 10:18:31.189191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:31.189191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:31.189191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:31.189191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:31.189191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:31.189191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:31.189191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:31.189191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:31.189191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:31.189191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:31.189191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:31.189191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:31.189191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:31.189191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:31.189191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:31.190097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:31.190097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:31.190097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:31.190097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:31.190097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:31.190097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:31.190097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:31.190097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:31.190097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:31.190097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:31.190097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:31.190097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:31.190097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:31.190097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:31.190097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:31.190097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:31.190097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:31.190097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:31.191608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:31.191608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:31.191608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:31.191608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:31.191608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:31.191608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:31.191608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:31.191608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:31.191608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 10:18:31.191608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:31.191608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 391/787] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:18:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:31.198542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579e1c4-e744-4d65-bee0-a51a6c7be8dc None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:31.233449 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:31.351671 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579e1c4-e744-4d65-bee0-a51a6c7be8dc None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:31.403620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a96df-09ee-47a2-b845-e37c4f7e67d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:31.404154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a96df-09ee-47a2-b845-e37c4f7e67d4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:31.404431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a96df-09ee-47a2-b845-e37c4f7e67d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:31.404707 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a96df-09ee-47a2-b845-e37c4f7e67d4 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:31.405468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a96df-09ee-47a2-b845-e37c4f7e67d4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:31.405771 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a96df-09ee-47a2-b845-e37c4f7e67d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:31.407185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a96df-09ee-47a2-b845-e37c4f7e67d4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:31.431793 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f657f-1b07-46f3-a87a-4301e14432b3 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:31.432132 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f657f-1b07-46f3-a87a-4301e14432b3 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:31.432330 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f657f-1b07-46f3-a87a-4301e14432b3 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:31.433682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 392/788] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:31.439635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13ba7f-b88c-40e9-a9ed-ce2d748f9d9c None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:31.439914 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13ba7f-b88c-40e9-a9ed-ce2d748f9d9c None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:31.440189 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13ba7f-b88c-40e9-a9ed-ce2d748f9d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:31.445594 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 397/789] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Sep 19 10:18:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:31.451631 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a96df-09ee-47a2-b845-e37c4f7e67d4 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:31.461306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-3f13ba7f-b88c-40e9-a9ed-ce2d748f9d9c None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:31.462531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-3f13ba7f-b88c-40e9-a9ed-ce2d748f9d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:31.521458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:31.543250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 393/790] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:18:31.601427 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:31.655880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a96df-09ee-47a2-b845-e37c4f7e67d4 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:31.703248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9866b-9ccf-47b0-bd8b-53493d9850c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:31.703885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9866b-9ccf-47b0-bd8b-53493d9850c7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:31.704164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9866b-9ccf-47b0-bd8b-53493d9850c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:31.704438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9866b-9ccf-47b0-bd8b-53493d9850c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:31.722990 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9866b-9ccf-47b0-bd8b-53493d9850c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:31.724433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9866b-9ccf-47b0-bd8b-53493d9850c7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:31.740695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-f5b9866b-9ccf-47b0-bd8b-53493d9850c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:31.740695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:31.740695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:31.740695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:31.740695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:31.740695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:31.740695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:31.740695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:31.740695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:31.740695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:31.740695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:31.740695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:31.740695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:31.740695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:31.740695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:31.740695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:31.742910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:31.742910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:31.742910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:31.742910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:31.742910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:31.742910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:31.742910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:31.742910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:31.742910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:31.742910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:31.742910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:31.742910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:31.742910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:31.742910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:31.742910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:31.742910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:31.742910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:31.745004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:31.745004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:31.745004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:31.745004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 398/791] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:18:31.751102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13ba7f-b88c-40e9-a9ed-ce2d748f9d9c None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:31.883150 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13ba7f-b88c-40e9-a9ed-ce2d748f9d9c None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:31.896860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9866b-9ccf-47b0-bd8b-53493d9850c7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:31.924782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d971e16-b96d-495f-9dec-76e1ce6d6c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:31.925444 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d971e16-b96d-495f-9dec-76e1ce6d6c78 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:31.925720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d971e16-b96d-495f-9dec-76e1ce6d6c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:31.925996 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d971e16-b96d-495f-9dec-76e1ce6d6c78 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:31.926656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d971e16-b96d-495f-9dec-76e1ce6d6c78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:31.926962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d971e16-b96d-495f-9dec-76e1ce6d6c78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:31.928384 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d971e16-b96d-495f-9dec-76e1ce6d6c78 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:31.942598 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 394/792] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:31.948792 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d971e16-b96d-495f-9dec-76e1ce6d6c78 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:32.011409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:32.062770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d971e16-b96d-495f-9dec-76e1ce6d6c78 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:32.105131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172270d-9513-42a5-a3cf-57ecece33323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:32.105875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172270d-9513-42a5-a3cf-57ecece33323 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:32.106583 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172270d-9513-42a5-a3cf-57ecece33323 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:32.106848 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172270d-9513-42a5-a3cf-57ecece33323 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:32.115754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172270d-9513-42a5-a3cf-57ecece33323 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:32.117777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172270d-9513-42a5-a3cf-57ecece33323 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:32.129801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9866b-9ccf-47b0-bd8b-53493d9850c7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:32.133859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-2172270d-9513-42a5-a3cf-57ecece33323 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:32.133859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:32.133859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:32.133859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:32.133859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:32.133859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:32.133859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:32.133859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:32.133859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:32.133859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:32.133859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:32.133859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:32.133859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:32.133859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:32.133859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:32.136434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:32.136434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:32.136434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:32.136434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:32.136434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:32.136434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:32.136434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:32.136434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:32.136434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:32.136434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:32.136434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:32.136434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:32.136434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:32.136434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:32.136434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:32.136434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:32.136434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:32.138082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:32.138082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:32.138082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:32.138082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:32.138082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:32.138082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:32.138082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:32.138082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:32.138082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 395/793] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:18:32.138082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89116d2-8ac5-43e1-a72c-bca018c72af6 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:32.138082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89116d2-8ac5-43e1-a72c-bca018c72af6 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:32.139248 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89116d2-8ac5-43e1-a72c-bca018c72af6 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:32.141095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 396/794] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:32.145047 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89116d2-8ac5-43e1-a72c-bca018c72af6 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:32.158283 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:32.181575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f366c-d325-4671-8c60-6c95bc73125a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:32.182139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f366c-d325-4671-8c60-6c95bc73125a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:32.182429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f366c-d325-4671-8c60-6c95bc73125a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:32.182685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f366c-d325-4671-8c60-6c95bc73125a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:32.194729 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f366c-d325-4671-8c60-6c95bc73125a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:32.200087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-822f366c-d325-4671-8c60-6c95bc73125a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:18:32.200087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:32.200087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:32.200087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:32.200087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:32.200087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:32.200087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:32.200087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:32.200087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:32.200087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:32.200087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:32.200087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:32.200087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:32.200087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:32.200087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:32.200087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:32.201087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:32.201087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:32.201087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:32.201087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:32.201087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:32.201087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:32.201087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:32.201087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:32.201087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:32.201087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:32.201087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:32.201087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:32.201087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:32.201087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:32.201087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:32.201087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:32.201954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:32.201954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:32.201954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:32.201954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:32.201954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:32.201954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:32.201954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:32.201954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:32.201954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:32.201954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:32.201954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:18:32.201954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:32.201954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 399/795] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:18:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:32.202668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30baed68-0103-4d93-a93e-e1d0493ea6fc None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:32.203275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30baed68-0103-4d93-a93e-e1d0493ea6fc None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:32.203275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30baed68-0103-4d93-a93e-e1d0493ea6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:32.223691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-30baed68-0103-4d93-a93e-e1d0493ea6fc None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:32.224834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-30baed68-0103-4d93-a93e-e1d0493ea6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:32.290342 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89116d2-8ac5-43e1-a72c-bca018c72af6 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:32.292870 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:32.305238 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 400/796] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:32.306797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30baed68-0103-4d93-a93e-e1d0493ea6fc None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:32.352766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080b3b1-422d-4c76-95b8-a24bbca16b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:32.353723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080b3b1-422d-4c76-95b8-a24bbca16b15 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:32.353999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080b3b1-422d-4c76-95b8-a24bbca16b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:32.354269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080b3b1-422d-4c76-95b8-a24bbca16b15 None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:32.354983 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080b3b1-422d-4c76-95b8-a24bbca16b15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:32.355287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080b3b1-422d-4c76-95b8-a24bbca16b15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:32.357047 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080b3b1-422d-4c76-95b8-a24bbca16b15 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:32.387099 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 397/797] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:32.392829 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080b3b1-422d-4c76-95b8-a24bbca16b15 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:32.468385 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30baed68-0103-4d93-a93e-e1d0493ea6fc None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:32.480198 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:32.504879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8a825-2472-443e-99e0-0f23ea07a675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:32.505517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8a825-2472-443e-99e0-0f23ea07a675 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:32.505909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8a825-2472-443e-99e0-0f23ea07a675 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:32.506182 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8a825-2472-443e-99e0-0f23ea07a675 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:32.506996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8a825-2472-443e-99e0-0f23ea07a675 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:32.507269 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8a825-2472-443e-99e0-0f23ea07a675 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:32.508622 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8a825-2472-443e-99e0-0f23ea07a675 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:32.522981 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 401/798] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:18:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:32.522981 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6afd11-ffdc-4383-91e2-a2c94571a46c None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:32.522981 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6afd11-ffdc-4383-91e2-a2c94571a46c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:32.522981 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6afd11-ffdc-4383-91e2-a2c94571a46c None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:32.522981 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 402/799] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:32.523522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6afd11-ffdc-4383-91e2-a2c94571a46c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:32.545908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080b3b1-422d-4c76-95b8-a24bbca16b15 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:32.586979 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b8930-08f8-4445-b561-e8704be75f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:32.587676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b8930-08f8-4445-b561-e8704be75f96 None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:32.588006 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b8930-08f8-4445-b561-e8704be75f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:32.588281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b8930-08f8-4445-b561-e8704be75f96 None admin] PATH_INFO: `/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/17a8bc6c506c45be897bda23eaab485a/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:32.590499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b8930-08f8-4445-b561-e8704be75f96 None admin] RBAC: Authorizing `identity:create_grant(project_id=e45b8fbe79994e7585095328a7ec9260, user_id=17a8bc6c506c45be897bda23eaab485a, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:32.591978 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b8930-08f8-4445-b561-e8704be75f96 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:32.635607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6afd11-ffdc-4383-91e2-a2c94571a46c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:32.668435 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41644c81-dc37-4bd6-8267-7c14941fdf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:32.669068 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41644c81-dc37-4bd6-8267-7c14941fdf01 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:32.669358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41644c81-dc37-4bd6-8267-7c14941fdf01 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:32.669633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41644c81-dc37-4bd6-8267-7c14941fdf01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:32.689090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 398/800] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:18:32 2022] PUT /identity/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/17a8bc6c506c45be897bda23eaab485a/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:18:32.689090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138b882-fe95-46f9-a783-95512266c8f3 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:32.689090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138b882-fe95-46f9-a783-95512266c8f3 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:32.689090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138b882-fe95-46f9-a783-95512266c8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:32.692672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41644c81-dc37-4bd6-8267-7c14941fdf01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:32.694434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41644c81-dc37-4bd6-8267-7c14941fdf01 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:32.705249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-3138b882-fe95-46f9-a783-95512266c8f3 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:32.706880 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-3138b882-fe95-46f9-a783-95512266c8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:32.715790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-41644c81-dc37-4bd6-8267-7c14941fdf01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:32.715790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:32.715790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:32.715790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:32.715790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:32.715790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:32.715790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:32.715790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:32.715790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:32.715790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:32.715790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:32.715790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:32.715790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:32.715790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:32.715790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:32.715790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:32.716844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:32.716844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:32.716844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:32.716844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:32.716844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:32.716844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:32.716844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:32.716844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:32.716844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:32.716844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:32.716844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:32.716844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:32.716844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:32.716844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:32.716844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:32.716844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:32.716844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:32.717874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:32.717874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:32.717874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:32.718586 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 403/801] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:18:32.722058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41644c81-dc37-4bd6-8267-7c14941fdf01 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:32.732701 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:32.758434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:32.795295 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 399/802] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:32.796183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138b882-fe95-46f9-a783-95512266c8f3 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:32.908993 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41644c81-dc37-4bd6-8267-7c14941fdf01 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:32.962579 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3186fe-5dae-49d8-907d-210e90aa8c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:32.963591 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3186fe-5dae-49d8-907d-210e90aa8c0b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:32.964070 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3186fe-5dae-49d8-907d-210e90aa8c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:32.964539 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3186fe-5dae-49d8-907d-210e90aa8c0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:32.970599 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138b882-fe95-46f9-a783-95512266c8f3 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:32.979681 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3186fe-5dae-49d8-907d-210e90aa8c0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:32.984818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-ff3186fe-5dae-49d8-907d-210e90aa8c0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:32.984818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:32.984818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:32.984818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:32.984818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:32.984818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:32.984818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:32.984818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:32.984818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:32.984818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:32.984818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:32.984818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:32.984818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:32.984818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:32.984818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:32.984818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:32.985775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:32.985775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:32.985775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:32.985775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:32.985775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:32.985775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:32.985775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:32.985775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:32.985775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:32.985775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:32.985775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:32.985775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:32.985775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:32.985775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:32.985775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:32.985775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:32.986628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:32.986628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:32.986628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:32.986628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:32.986628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:32.986628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:32.986628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:32.986628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:32.986628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:32.986628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:32.986628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:32.986628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:32.989474 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 404/803] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:18:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:33.000122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3186fe-5dae-49d8-907d-210e90aa8c0b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:33.017085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05101f9-58c3-4e55-929b-430152a4c15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:33.017085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05101f9-58c3-4e55-929b-430152a4c15f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:33.017085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05101f9-58c3-4e55-929b-430152a4c15f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:33.017085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05101f9-58c3-4e55-929b-430152a4c15f None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:33.017779 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05101f9-58c3-4e55-929b-430152a4c15f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:33.017779 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05101f9-58c3-4e55-929b-430152a4c15f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:33.017779 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05101f9-58c3-4e55-929b-430152a4c15f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:33.039610 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 400/804] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:33.045832 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05101f9-58c3-4e55-929b-430152a4c15f None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:33.075111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:33.162804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:33.182342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3186fe-5dae-49d8-907d-210e90aa8c0b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:33.196492 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05101f9-58c3-4e55-929b-430152a4c15f None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:33.224263 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556471f-9de7-421c-87eb-43c2022b5ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:33.225092 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556471f-9de7-421c-87eb-43c2022b5ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:33.225387 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556471f-9de7-421c-87eb-43c2022b5ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:33.225664 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556471f-9de7-421c-87eb-43c2022b5ee7 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:33.226436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556471f-9de7-421c-87eb-43c2022b5ee7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:33.226839 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556471f-9de7-421c-87eb-43c2022b5ee7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:33.228257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556471f-9de7-421c-87eb-43c2022b5ee7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:33.239504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 405/805] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:18:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:33.240823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556471f-9de7-421c-87eb-43c2022b5ee7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:33.247084 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7643081-f6c3-4a2d-bd0b-94d1f8b60bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:33.247724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7643081-f6c3-4a2d-bd0b-94d1f8b60bca None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:33.248052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7643081-f6c3-4a2d-bd0b-94d1f8b60bca None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:33.248326 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7643081-f6c3-4a2d-bd0b-94d1f8b60bca None admin] PATH_INFO: `/v3/users/nova` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:33.263968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7643081-f6c3-4a2d-bd0b-94d1f8b60bca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:33.266067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7643081-f6c3-4a2d-bd0b-94d1f8b60bca None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:33.282553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-b7643081-f6c3-4a2d-bd0b-94d1f8b60bca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 10:18:33.282553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:33.282553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:33.282553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:33.282553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:33.282553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:33.282553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:33.282553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:33.282553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:33.282553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:33.282553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:33.282553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:33.282553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:33.282553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:33.282553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:33.282553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:33.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:33.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:33.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:33.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:33.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:33.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:33.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:33.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:33.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:33.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:33.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:33.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:33.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:33.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:33.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:33.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:33.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:33.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:33.284523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:33.284523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:33.284523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:33.284523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:33.284523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:33.284523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:33.284523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:33.284523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:33.284523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 10:18:33.284523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:33.284523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 401/806] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Sep 19 10:18:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:33.285178 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7643081-f6c3-4a2d-bd0b-94d1f8b60bca None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:33.359654 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:33.361128 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556471f-9de7-421c-87eb-43c2022b5ee7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:33.391835 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7643081-f6c3-4a2d-bd0b-94d1f8b60bca None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:33.397184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d2e3e-a355-47ba-8cbb-51e54e4dd6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:33.398090 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d2e3e-a355-47ba-8cbb-51e54e4dd6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:33.398358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d2e3e-a355-47ba-8cbb-51e54e4dd6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:33.398631 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d2e3e-a355-47ba-8cbb-51e54e4dd6e0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:33.399222 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d2e3e-a355-47ba-8cbb-51e54e4dd6e0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:33.401158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d2e3e-a355-47ba-8cbb-51e54e4dd6e0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:33.410442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-574d2e3e-a355-47ba-8cbb-51e54e4dd6e0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 10:18:33.410442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:33.410442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:33.410442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:33.410442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:33.410442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:33.410442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:33.410442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:33.410442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 10:18:33.410442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:33.410442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:33.410442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:33.410442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:33.410442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:33.410442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:33.410442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:33.410442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:33.411437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:33.411437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:33.411437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:33.411437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:33.411437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:33.411437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:33.411437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:33.411437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:33.411437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:33.411437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:33.411437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:33.411437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:33.411437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:33.411437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:33.411437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:33.411437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:33.412286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:33.412286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:33.412286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:33.412286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:33.412286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:33.412286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:33.412286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:33.412286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:33.412286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:33.412286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:33.412286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:33.412286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:33.412286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 10:18:33.412286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:33.417023 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 406/807] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Sep 19 10:18:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:33.418419 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d858b76-c5bb-4ce9-b760-1541d18d2c67 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:33.418716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d858b76-c5bb-4ce9-b760-1541d18d2c67 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:33.419001 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d858b76-c5bb-4ce9-b760-1541d18d2c67 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:33.424817 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 407/808] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:33.425633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d858b76-c5bb-4ce9-b760-1541d18d2c67 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:33.430865 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dc005-93b3-4bb6-9ee9-ee028568f884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:33.431608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dc005-93b3-4bb6-9ee9-ee028568f884 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:33.431889 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dc005-93b3-4bb6-9ee9-ee028568f884 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:33.432160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dc005-93b3-4bb6-9ee9-ee028568f884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:33.441410 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:33.444386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016dc005-93b3-4bb6-9ee9-ee028568f884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:33.445867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016dc005-93b3-4bb6-9ee9-ee028568f884 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:33.461279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-016dc005-93b3-4bb6-9ee9-ee028568f884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:33.461279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:33.461279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:33.461279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:33.461279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:33.461279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:33.461279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:33.461279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:33.461279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:33.461279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:33.461279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:33.461279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:33.461279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:33.461279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:33.461279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:33.461279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:33.462368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:33.462368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:33.462368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:33.462368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:33.462368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:33.462368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:33.462368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:33.462368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:33.462368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:33.462368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:33.462368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:33.462368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:33.462368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:33.462368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:33.462368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:33.462368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:33.462368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:33.464384 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:33.464384 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:33.464384 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:33.464384 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 402/809] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:33.464384 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dc005-93b3-4bb6-9ee9-ee028568f884 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:33.553333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d858b76-c5bb-4ce9-b760-1541d18d2c67 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:33.567109 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dc005-93b3-4bb6-9ee9-ee028568f884 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:33.594095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:33.594295 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bef2e-1b72-4936-952a-78f664a80373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:33.594825 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bef2e-1b72-4936-952a-78f664a80373 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:33.595140 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bef2e-1b72-4936-952a-78f664a80373 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:33.595413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bef2e-1b72-4936-952a-78f664a80373 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:33.596153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2bef2e-1b72-4936-952a-78f664a80373 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:33.596462 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2bef2e-1b72-4936-952a-78f664a80373 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:33.597933 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2bef2e-1b72-4936-952a-78f664a80373 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:33.608227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51eb9c3-fb13-41d6-a0c5-253d9ff53dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:33.608833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51eb9c3-fb13-41d6-a0c5-253d9ff53dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:33.609135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51eb9c3-fb13-41d6-a0c5-253d9ff53dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:33.609408 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51eb9c3-fb13-41d6-a0c5-253d9ff53dd0 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:33.610033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51eb9c3-fb13-41d6-a0c5-253d9ff53dd0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:33.610411 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51eb9c3-fb13-41d6-a0c5-253d9ff53dd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:33.611747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51eb9c3-fb13-41d6-a0c5-253d9ff53dd0 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:33.621696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 403/810] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Sep 19 10:18:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:33.621696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c39611-373b-473a-9772-20839b3b9793 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:33.621696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c39611-373b-473a-9772-20839b3b9793 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:33.621696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c39611-373b-473a-9772-20839b3b9793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:33.622075 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 408/811] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Sep 19 10:18:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:33.622075 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bef2e-1b72-4936-952a-78f664a80373 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:33.638478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-c7c39611-373b-473a-9772-20839b3b9793 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:33.640436 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c7c39611-373b-473a-9772-20839b3b9793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:33.697024 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:33.713809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 404/812] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:33.714656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c39611-373b-473a-9772-20839b3b9793 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:33.744098 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bef2e-1b72-4936-952a-78f664a80373 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:33.778353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8003da2-19b1-4f2d-a53d-1e4b397dda62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:33.778925 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8003da2-19b1-4f2d-a53d-1e4b397dda62 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:33.779208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8003da2-19b1-4f2d-a53d-1e4b397dda62 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:33.779487 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8003da2-19b1-4f2d-a53d-1e4b397dda62 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:33.780125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8003da2-19b1-4f2d-a53d-1e4b397dda62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:33.780427 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8003da2-19b1-4f2d-a53d-1e4b397dda62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:33.781803 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8003da2-19b1-4f2d-a53d-1e4b397dda62 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:33.794150 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 409/813] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:33.799364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8003da2-19b1-4f2d-a53d-1e4b397dda62 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:33.846518 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c39611-373b-473a-9772-20839b3b9793 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:33.873410 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:33.886085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b51bd-345d-4afb-ab85-2affcc829380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:33.886704 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b51bd-345d-4afb-ab85-2affcc829380 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:33.886980 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b51bd-345d-4afb-ab85-2affcc829380 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:33.887301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b51bd-345d-4afb-ab85-2affcc829380 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:33.900997 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8003da2-19b1-4f2d-a53d-1e4b397dda62 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:33.928213 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:33.937621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde6783-47d1-4208-9bf9-f66defa02a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:33.938227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde6783-47d1-4208-9bf9-f66defa02a02 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:33.938509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde6783-47d1-4208-9bf9-f66defa02a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:33.938784 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde6783-47d1-4208-9bf9-f66defa02a02 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:33.939415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde6783-47d1-4208-9bf9-f66defa02a02 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:33.939716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde6783-47d1-4208-9bf9-f66defa02a02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:33.941208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde6783-47d1-4208-9bf9-f66defa02a02 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:33.946499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 410/814] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Sep 19 10:18:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:33.947859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde6783-47d1-4208-9bf9-f66defa02a02 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:33.963166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b51bd-345d-4afb-ab85-2affcc829380 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:33.964672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b51bd-345d-4afb-ab85-2affcc829380 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:34.050961 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde6783-47d1-4208-9bf9-f66defa02a02 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:34.077298 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 405/815] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:18:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:18:34.078082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b51bd-345d-4afb-ab85-2affcc829380 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:34.085695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fec84-3c1a-4cab-a03e-76cca1f896cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:34.086322 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fec84-3c1a-4cab-a03e-76cca1f896cb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:34.086600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fec84-3c1a-4cab-a03e-76cca1f896cb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:34.086872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fec84-3c1a-4cab-a03e-76cca1f896cb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:34.095254 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fec84-3c1a-4cab-a03e-76cca1f896cb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:34.096682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fec84-3c1a-4cab-a03e-76cca1f896cb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:34.096875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:34.108392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-cf0fec84-3c1a-4cab-a03e-76cca1f896cb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 10:18:34.108392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:34.108392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:34.108392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:34.108392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:34.108392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:34.108392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:34.108392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:34.108392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:34.108392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:34.108392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:34.108392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:34.108392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:34.108392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:34.108392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:34.108392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:34.114019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:34.114019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:34.114019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:34.114019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:34.114019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:34.114019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:34.114019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:34.114019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:34.114019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:34.114019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:34.114019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:34.114019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:34.114019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:34.114019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:34.114019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:34.114019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:34.114019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:34.114019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:34.115158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:34.115158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:34.115158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:34.115158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:34.115158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:34.115158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:34.115158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:34.115158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:34.115158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 10:18:34.115158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:34.115158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 411/816] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 19 10:18:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:34.115158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fec84-3c1a-4cab-a03e-76cca1f896cb None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:34.171340 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:34.187014 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b51bd-345d-4afb-ab85-2affcc829380 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:34.221662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fec84-3c1a-4cab-a03e-76cca1f896cb None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:34.227296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c038802-53ef-43c3-ada7-6979ed17122e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:34.227975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c038802-53ef-43c3-ada7-6979ed17122e None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:34.228348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c038802-53ef-43c3-ada7-6979ed17122e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:34.228702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c038802-53ef-43c3-ada7-6979ed17122e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:34.229489 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c038802-53ef-43c3-ada7-6979ed17122e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:34.230924 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c038802-53ef-43c3-ada7-6979ed17122e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:34.254833 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ad036-7000-4446-82aa-212f11d35c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:34.256109 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ad036-7000-4446-82aa-212f11d35c31 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:34.256109 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ad036-7000-4446-82aa-212f11d35c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:34.256109 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ad036-7000-4446-82aa-212f11d35c31 None admin] PATH_INFO: `/v3/services/network` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:34.256519 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ad036-7000-4446-82aa-212f11d35c31 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:34.257874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ad036-7000-4446-82aa-212f11d35c31 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:34.266809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-008ad036-7000-4446-82aa-212f11d35c31 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 10:18:34.266809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:34.266809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:34.266809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:34.266809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:34.266809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:34.266809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:34.266809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:34.266809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 10:18:34.266809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:34.266809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:34.266809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:34.266809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:34.266809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:34.266809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:34.266809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:34.266809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:34.267973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:34.267973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:34.267973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:34.267973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:34.267973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:34.267973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:34.267973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:34.267973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:34.267973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:34.267973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:34.267973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:34.267973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:34.267973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:34.267973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:34.267973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:34.267973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:34.268805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:34.268805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:34.268805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:34.268805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:34.268805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:34.268805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:34.268805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:34.268805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:34.268805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:34.268805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:34.268805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:34.268805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:34.268805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 10:18:34.268805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:34.268805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 412/817] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:34 2022] GET /identity/v3/services/network => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:34.269751 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ad036-7000-4446-82aa-212f11d35c31 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:34.329396 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 406/818] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:18:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:18:34.330378 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c038802-53ef-43c3-ada7-6979ed17122e None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:34.386193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ad036-7000-4446-82aa-212f11d35c31 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:34.410464 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:34.452202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb74b68-fb3b-4efc-9d8d-a883a595d9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:34.453035 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb74b68-fb3b-4efc-9d8d-a883a595d9bd None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:34.453636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb74b68-fb3b-4efc-9d8d-a883a595d9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:34.454012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb74b68-fb3b-4efc-9d8d-a883a595d9bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:34.458453 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:34.467203 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb74b68-fb3b-4efc-9d8d-a883a595d9bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:34.469558 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb74b68-fb3b-4efc-9d8d-a883a595d9bd None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:34.488054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-3eb74b68-fb3b-4efc-9d8d-a883a595d9bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:34.488054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:34.488054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:34.488054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:34.488054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:34.488054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:34.488054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:34.488054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:34.488054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:34.488054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:34.488054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:34.488054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:34.488054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:34.488054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:34.488054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:34.488054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:34.489296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:34.489296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:34.489296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:34.489296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:34.489296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:34.489296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:34.489296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:34.489296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:34.489296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:34.489296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:34.489296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:34.489296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:34.489296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:34.489296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:34.489296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:34.489296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:34.489296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:34.490509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:34.490509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:34.490509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:34.496675 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 413/819] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:34.496675 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb74b68-fb3b-4efc-9d8d-a883a595d9bd None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:34.502309 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c038802-53ef-43c3-ada7-6979ed17122e None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:34.551505 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05823885-c8e5-4f95-8825-c1f04fd338a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:34.552726 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05823885-c8e5-4f95-8825-c1f04fd338a6 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:34.553881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05823885-c8e5-4f95-8825-c1f04fd338a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:34.553881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05823885-c8e5-4f95-8825-c1f04fd338a6 None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:34.554287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05823885-c8e5-4f95-8825-c1f04fd338a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:34.554741 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05823885-c8e5-4f95-8825-c1f04fd338a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:34.556497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05823885-c8e5-4f95-8825-c1f04fd338a6 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:34.581537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 407/820] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Sep 19 10:18:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:34.582312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05823885-c8e5-4f95-8825-c1f04fd338a6 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:34.627914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb74b68-fb3b-4efc-9d8d-a883a595d9bd None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:34.666933 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73660e5d-7ff9-406e-a4e1-3ca70f24003f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:34.667539 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73660e5d-7ff9-406e-a4e1-3ca70f24003f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:34.668388 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73660e5d-7ff9-406e-a4e1-3ca70f24003f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:34.668673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73660e5d-7ff9-406e-a4e1-3ca70f24003f None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:34.669341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73660e5d-7ff9-406e-a4e1-3ca70f24003f None admin] RBAC: Adding query filter params (name=network) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:34.669650 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73660e5d-7ff9-406e-a4e1-3ca70f24003f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:34.671901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73660e5d-7ff9-406e-a4e1-3ca70f24003f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:34.677123 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 414/821] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 19 10:18:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:34.682949 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73660e5d-7ff9-406e-a4e1-3ca70f24003f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:34.699419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05823885-c8e5-4f95-8825-c1f04fd338a6 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:34.720097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:34.732454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:34.737669 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01203692-27b2-4401-a467-5f15081164b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:34.738590 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01203692-27b2-4401-a467-5f15081164b5 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:34.738969 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01203692-27b2-4401-a467-5f15081164b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:34.739351 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01203692-27b2-4401-a467-5f15081164b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:34.740102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01203692-27b2-4401-a467-5f15081164b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:34.740520 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01203692-27b2-4401-a467-5f15081164b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:34.742005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01203692-27b2-4401-a467-5f15081164b5 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:34.758891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 408/822] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:34.760620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01203692-27b2-4401-a467-5f15081164b5 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:34.810686 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73660e5d-7ff9-406e-a4e1-3ca70f24003f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:34.853702 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f7ace-c128-493b-ad50-8744a4eeac49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:34.854423 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f7ace-c128-493b-ad50-8744a4eeac49 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:34.854798 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f7ace-c128-493b-ad50-8744a4eeac49 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:34.855171 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f7ace-c128-493b-ad50-8744a4eeac49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:34.869624 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f7ace-c128-493b-ad50-8744a4eeac49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:34.872140 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f7ace-c128-493b-ad50-8744a4eeac49 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:34.891528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01203692-27b2-4401-a467-5f15081164b5 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:34.891845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-fd5f7ace-c128-493b-ad50-8744a4eeac49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:34.891845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:34.891845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:34.891845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:34.891845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:34.891845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:34.891845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:34.891845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:34.891845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:34.891845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:34.891845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:34.891845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:34.891845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:34.891845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:34.891845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:34.891845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:34.892974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:34.892974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:34.892974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:34.892974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:34.892974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:34.892974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:34.892974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:34.892974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:34.892974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:34.892974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:34.892974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:34.892974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:34.892974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:34.892974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:34.892974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:34.892974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:34.892974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:34.894121 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:34.894121 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:34.894121 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:34.895274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 415/823] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:34.896598 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f7ace-c128-493b-ad50-8744a4eeac49 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:34.940286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e25ce0-3f90-4c74-891c-8ff8e63a261d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:34.942288 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e25ce0-3f90-4c74-891c-8ff8e63a261d None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:34.942749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e25ce0-3f90-4c74-891c-8ff8e63a261d None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:34.943163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e25ce0-3f90-4c74-891c-8ff8e63a261d None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:34.943929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e25ce0-3f90-4c74-891c-8ff8e63a261d None admin] RBAC: Adding query filter params (type=network) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:34.944373 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e25ce0-3f90-4c74-891c-8ff8e63a261d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:34.945913 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e25ce0-3f90-4c74-891c-8ff8e63a261d None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:34.953366 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 409/824] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 19 10:18:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:34.954497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e25ce0-3f90-4c74-891c-8ff8e63a261d None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:34.996110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:35.016219 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f7ace-c128-493b-ad50-8744a4eeac49 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:35.022152 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:35.058065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d06a3-d076-4ae6-a614-1778ba9db8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:35.058690 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d06a3-d076-4ae6-a614-1778ba9db8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:35.059120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d06a3-d076-4ae6-a614-1778ba9db8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:35.059402 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d06a3-d076-4ae6-a614-1778ba9db8b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:35.065484 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e25ce0-3f90-4c74-891c-8ff8e63a261d None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:35.066747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d06a3-d076-4ae6-a614-1778ba9db8b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:35.068181 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d06a3-d076-4ae6-a614-1778ba9db8b7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:35.079516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-c56d06a3-d076-4ae6-a614-1778ba9db8b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:35.079516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:35.079516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:35.079516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:35.079516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:35.079516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:35.079516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:35.079516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:35.079516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:35.079516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:35.079516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:35.079516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:35.079516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:35.079516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:35.079516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:35.080608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:35.080608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:35.080608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:35.080608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:35.080608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:35.080608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:35.080608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:35.080608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:35.080608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:35.080608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:35.080608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:35.080608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:35.080608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:35.080608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:35.080608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:35.080608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:35.080608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:35.081575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:35.081575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:35.081575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:35.081575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:35.081575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:35.081575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:35.081575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:35.081575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:35.081575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 416/825] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:35.082065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d06a3-d076-4ae6-a614-1778ba9db8b7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:35.108445 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc938e2d-cb46-4adf-86a0-ec59d47834bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:35.109290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc938e2d-cb46-4adf-86a0-ec59d47834bf None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:35.109574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc938e2d-cb46-4adf-86a0-ec59d47834bf None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:35.109846 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc938e2d-cb46-4adf-86a0-ec59d47834bf None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:35.110656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc938e2d-cb46-4adf-86a0-ec59d47834bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:35.111286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc938e2d-cb46-4adf-86a0-ec59d47834bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:35.113178 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc938e2d-cb46-4adf-86a0-ec59d47834bf None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:35.128082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 410/826] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:35.129634 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ef751-8698-4465-8c82-0eaa3bc344ec None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:35.129977 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ef751-8698-4465-8c82-0eaa3bc344ec None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:35.130317 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ef751-8698-4465-8c82-0eaa3bc344ec None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:35.131727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 411/827] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:35.132340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ef751-8698-4465-8c82-0eaa3bc344ec None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:35.198651 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d06a3-d076-4ae6-a614-1778ba9db8b7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:35.235399 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ef76e-3d20-4079-b9f3-d873fdeac280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:35.235911 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ef76e-3d20-4079-b9f3-d873fdeac280 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:35.236190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ef76e-3d20-4079-b9f3-d873fdeac280 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:35.236489 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ef76e-3d20-4079-b9f3-d873fdeac280 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:35.237155 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ef76e-3d20-4079-b9f3-d873fdeac280 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:35.238512 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ef76e-3d20-4079-b9f3-d873fdeac280 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:35.242498 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ef751-8698-4465-8c82-0eaa3bc344ec None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:35.249929 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:35.260754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:35.278191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686cbf1-5b84-4900-afa3-983169d1ee7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:35.278824 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686cbf1-5b84-4900-afa3-983169d1ee7c None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:35.279104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686cbf1-5b84-4900-afa3-983169d1ee7c None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:35.279375 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686cbf1-5b84-4900-afa3-983169d1ee7c None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:35.280059 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686cbf1-5b84-4900-afa3-983169d1ee7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:35.280364 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686cbf1-5b84-4900-afa3-983169d1ee7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:35.281822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686cbf1-5b84-4900-afa3-983169d1ee7c None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:35.292878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 412/828] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:35.294487 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686cbf1-5b84-4900-afa3-983169d1ee7c None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:35.335657 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 417/829] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:18:35 2022] POST /identity/v3/endpoints => generated 337 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:18:35.337169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2864a9-2218-455c-b0eb-a7291cc561bf None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:35.341249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2864a9-2218-455c-b0eb-a7291cc561bf None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:35.341539 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2864a9-2218-455c-b0eb-a7291cc561bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:35.370302 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8f2864a9-2218-455c-b0eb-a7291cc561bf None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:35.372113 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8f2864a9-2218-455c-b0eb-a7291cc561bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:35.408767 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686cbf1-5b84-4900-afa3-983169d1ee7c None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:35.457097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5881410-930d-426b-8306-408c662efac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:35.457961 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5881410-930d-426b-8306-408c662efac4 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:35.458313 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5881410-930d-426b-8306-408c662efac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:35.458684 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5881410-930d-426b-8306-408c662efac4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:35.466822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5881410-930d-426b-8306-408c662efac4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:35.468431 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5881410-930d-426b-8306-408c662efac4 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:35.474953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 418/830] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:35.474953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2864a9-2218-455c-b0eb-a7291cc561bf None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:35.492985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-f5881410-930d-426b-8306-408c662efac4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:35.492985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:35.492985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:35.492985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:35.492985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:35.492985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:35.492985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:35.492985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:35.492985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:35.492985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:35.492985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:35.492985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:35.492985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:35.492985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:35.492985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:35.494631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:35.494631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:35.494631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:35.494631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:35.494631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:35.494631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:35.494631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:35.494631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:35.494631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:35.494631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:35.494631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:35.494631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:35.494631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:35.494631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:35.494631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:35.494631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:35.494631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:35.495543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:35.495543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:35.495543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:35.495543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:35.495543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:35.495543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:35.495543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:35.495543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:35.495543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 413/831] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:35.504158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5881410-930d-426b-8306-408c662efac4 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:35.556167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:35.639086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2864a9-2218-455c-b0eb-a7291cc561bf None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:35.643349 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5881410-930d-426b-8306-408c662efac4 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:35.679124 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:35.683119 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8bf70-90e4-41c5-88da-6b9c34eef3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:35.683689 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8bf70-90e4-41c5-88da-6b9c34eef3cc None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:35.683970 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8bf70-90e4-41c5-88da-6b9c34eef3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:35.684263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8bf70-90e4-41c5-88da-6b9c34eef3cc None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:35.684935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8bf70-90e4-41c5-88da-6b9c34eef3cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:35.685342 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8bf70-90e4-41c5-88da-6b9c34eef3cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:35.687344 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8bf70-90e4-41c5-88da-6b9c34eef3cc None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:35.696543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7381e-e2dc-4096-ae8e-00f66341d860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:35.697969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7381e-e2dc-4096-ae8e-00f66341d860 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:35.698390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7381e-e2dc-4096-ae8e-00f66341d860 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:35.698805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7381e-e2dc-4096-ae8e-00f66341d860 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:35.699700 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7381e-e2dc-4096-ae8e-00f66341d860 None admin] RBAC: Adding query filter params (role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=b4007794a30247c4a318af62d05e3b92) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:35.700165 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7381e-e2dc-4096-ae8e-00f66341d860 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:35.700626 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 414/832] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:35.701987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7381e-e2dc-4096-ae8e-00f66341d860 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:35.706671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8bf70-90e4-41c5-88da-6b9c34eef3cc None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:35.709272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 419/833] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:18:35 2022] GET /identity/v3/role_assignments?user.id=b4007794a30247c4a318af62d05e3b92&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:35.812061 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7381e-e2dc-4096-ae8e-00f66341d860 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:35.823934 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:35.844736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8bf70-90e4-41c5-88da-6b9c34eef3cc None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:35.893017 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dc83d-d28a-40ce-b162-03157fa86342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:35.894086 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dc83d-d28a-40ce-b162-03157fa86342 None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:35.894371 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dc83d-d28a-40ce-b162-03157fa86342 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:35.894653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dc83d-d28a-40ce-b162-03157fa86342 None admin] PATH_INFO: `/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/4ae237601b4a4aab9e969fedb5ee37d7/roles/39be941a839d4eb6a2146c561c7a2851` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:35.910937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6dc83d-d28a-40ce-b162-03157fa86342 None admin] RBAC: Authorizing `identity:create_grant(project_id=e45b8fbe79994e7585095328a7ec9260, user_id=4ae237601b4a4aab9e969fedb5ee37d7, role_id=39be941a839d4eb6a2146c561c7a2851)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:35.912519 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6dc83d-d28a-40ce-b162-03157fa86342 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:35.930151 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7381e-e2dc-4096-ae8e-00f66341d860 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:35.967433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80473493-0e19-4f67-ad20-750cc5632637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:35.968065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80473493-0e19-4f67-ad20-750cc5632637 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:35.968346 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80473493-0e19-4f67-ad20-750cc5632637 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:35.968619 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80473493-0e19-4f67-ad20-750cc5632637 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:35.976607 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 415/834] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:18:35 2022] PUT /identity/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/4ae237601b4a4aab9e969fedb5ee37d7/roles/39be941a839d4eb6a2146c561c7a2851 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:18:35.980736 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80473493-0e19-4f67-ad20-750cc5632637 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:35.988054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-80473493-0e19-4f67-ad20-750cc5632637 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:18:35.988054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:35.988054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:35.988054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:35.988054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:35.988054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:35.988054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:35.988054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:35.988054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:35.988054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:35.988054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:35.988054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:35.988054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:35.988054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:35.988054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:35.988054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:35.989126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:35.989126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:35.989126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:35.989126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:35.989126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:35.989126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:35.989126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:35.989126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:35.989126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:35.989126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:35.989126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:35.989126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:35.989126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:35.989126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:35.989126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:35.989126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:35.990008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:35.990008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:35.990008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:35.990008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:35.990008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:35.990008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:35.990008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:35.990008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:35.990008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:35.990008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:35.990008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:18:35.990008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:35.990008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 420/835] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:18:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:35.995801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dc83d-d28a-40ce-b162-03157fa86342 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:36.115975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dc83d-d28a-40ce-b162-03157fa86342 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:36.158803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca25503-eead-42fe-b478-97976e0c65c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:36.159426 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca25503-eead-42fe-b478-97976e0c65c3 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:36.159707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca25503-eead-42fe-b478-97976e0c65c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:36.159984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca25503-eead-42fe-b478-97976e0c65c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:36.160655 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca25503-eead-42fe-b478-97976e0c65c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:36.161016 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca25503-eead-42fe-b478-97976e0c65c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:36.162455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca25503-eead-42fe-b478-97976e0c65c3 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:36.171408 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 416/836] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:18:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:36.177959 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80473493-0e19-4f67-ad20-750cc5632637 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:36.231407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:36.283577 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80473493-0e19-4f67-ad20-750cc5632637 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:36.319285 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf75e07-c922-4ffc-bcba-660038036559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:36.320000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf75e07-c922-4ffc-bcba-660038036559 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:36.320282 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf75e07-c922-4ffc-bcba-660038036559 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:36.320561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf75e07-c922-4ffc-bcba-660038036559 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:36.332027 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf75e07-c922-4ffc-bcba-660038036559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:36.333497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf75e07-c922-4ffc-bcba-660038036559 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:36.348493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-baf75e07-c922-4ffc-bcba-660038036559 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:36.348493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:36.348493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:36.348493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:36.348493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:36.348493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:36.348493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:36.348493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:36.348493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:36.348493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:36.348493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:36.348493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:36.348493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:36.348493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:36.348493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:36.348493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:36.349544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:36.349544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:36.349544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:36.349544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:36.349544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:36.349544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:36.349544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:36.349544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:36.349544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:36.349544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:36.349544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:36.349544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:36.349544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:36.349544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:36.349544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:36.349544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:36.349544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:36.350460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:36.350460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:36.350460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:36.350460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 421/837] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:18:36.355383 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca25503-eead-42fe-b478-97976e0c65c3 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:36.359297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:36.463453 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca25503-eead-42fe-b478-97976e0c65c3 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:36.497839 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46bbe3-8571-42ff-a4fd-29d068236c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:36.498458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46bbe3-8571-42ff-a4fd-29d068236c93 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:36.498796 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46bbe3-8571-42ff-a4fd-29d068236c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:36.499079 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46bbe3-8571-42ff-a4fd-29d068236c93 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:36.499724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46bbe3-8571-42ff-a4fd-29d068236c93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:36.500028 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46bbe3-8571-42ff-a4fd-29d068236c93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:36.501419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46bbe3-8571-42ff-a4fd-29d068236c93 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:36.514887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 417/838] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:36.520683 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf75e07-c922-4ffc-bcba-660038036559 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:36.632751 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf75e07-c922-4ffc-bcba-660038036559 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:36.634863 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:36.667279 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3ed6a-df5b-42f8-a5b1-6975c0c535e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:36.667895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3ed6a-df5b-42f8-a5b1-6975c0c535e7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:36.668168 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3ed6a-df5b-42f8-a5b1-6975c0c535e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:36.668444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3ed6a-df5b-42f8-a5b1-6975c0c535e7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:36.675834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3ed6a-df5b-42f8-a5b1-6975c0c535e7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:36.677297 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3ed6a-df5b-42f8-a5b1-6975c0c535e7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:36.689778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-07f3ed6a-df5b-42f8-a5b1-6975c0c535e7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 10:18:36.689778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:36.689778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:36.689778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:36.689778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:36.689778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:36.689778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:36.689778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:36.689778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:36.689778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:36.689778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:36.689778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:36.689778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:36.689778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:36.689778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:36.689778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:36.690778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:36.690778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:36.690778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:36.690778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:36.690778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:36.690778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:36.690778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:36.690778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:36.690778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:36.690778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:36.690778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:36.690778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:36.690778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:36.690778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:36.690778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:36.690778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:36.690778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:36.690778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:36.691701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:36.691701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:36.691701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:36.691701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:36.691701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:36.691701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:36.691701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:36.691701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:36.691701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 10:18:36.691701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:36.692241 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a5ff8-ed05-4c1c-a265-b46230ac4384 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:36.692241 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a5ff8-ed05-4c1c-a265-b46230ac4384 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:36.692241 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a5ff8-ed05-4c1c-a265-b46230ac4384 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:36.692437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 422/839] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:18:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:36.692503 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 418/840] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:18:36.697914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053f2cb-cc71-4dba-8fd9-616d5babd734 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:36.698221 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053f2cb-cc71-4dba-8fd9-616d5babd734 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:36.698454 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053f2cb-cc71-4dba-8fd9-616d5babd734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:36.699281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a5ff8-ed05-4c1c-a265-b46230ac4384 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:36.720588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-0053f2cb-cc71-4dba-8fd9-616d5babd734 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:36.721892 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0053f2cb-cc71-4dba-8fd9-616d5babd734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:36.782534 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:36.805607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 423/841] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:36.841361 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a5ff8-ed05-4c1c-a265-b46230ac4384 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:36.879703 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d1410-2bc3-44cd-8953-7480ff9e98ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:36.880311 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d1410-2bc3-44cd-8953-7480ff9e98ed None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:36.880587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d1410-2bc3-44cd-8953-7480ff9e98ed None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:36.880861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d1410-2bc3-44cd-8953-7480ff9e98ed None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:36.881620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d1410-2bc3-44cd-8953-7480ff9e98ed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:36.881923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d1410-2bc3-44cd-8953-7480ff9e98ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:36.883484 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d1410-2bc3-44cd-8953-7480ff9e98ed None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:36.909991 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 419/842] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Sep 19 10:18:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:36.915361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053f2cb-cc71-4dba-8fd9-616d5babd734 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:37.051778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053f2cb-cc71-4dba-8fd9-616d5babd734 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:37.053378 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:37.088001 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad14ab-2b22-4fe3-8b32-a5e237c8bb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:37.088612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad14ab-2b22-4fe3-8b32-a5e237c8bb08 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:37.088912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad14ab-2b22-4fe3-8b32-a5e237c8bb08 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:37.089239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad14ab-2b22-4fe3-8b32-a5e237c8bb08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:37.113405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d1410-2bc3-44cd-8953-7480ff9e98ed None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:37.158583 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad14ab-2b22-4fe3-8b32-a5e237c8bb08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:37.160060 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad14ab-2b22-4fe3-8b32-a5e237c8bb08 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:37.255380 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:37.270173 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 424/843] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:18:36 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:18:37.270723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad14ab-2b22-4fe3-8b32-a5e237c8bb08 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:37.330819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d1410-2bc3-44cd-8953-7480ff9e98ed None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:37.374515 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad14ab-2b22-4fe3-8b32-a5e237c8bb08 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:37.383048 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:37.396732 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9678a6-f33c-4758-a487-0e968dd64b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:37.403102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9678a6-f33c-4758-a487-0e968dd64b5c None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:37.404822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9678a6-f33c-4758-a487-0e968dd64b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:37.405553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9678a6-f33c-4758-a487-0e968dd64b5c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:37.406833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9678a6-f33c-4758-a487-0e968dd64b5c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:37.409584 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9678a6-f33c-4758-a487-0e968dd64b5c None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:37.410972 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19ee0e-afa5-46d4-8806-b0ba1dfcd990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:37.411649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19ee0e-afa5-46d4-8806-b0ba1dfcd990 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:37.411930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19ee0e-afa5-46d4-8806-b0ba1dfcd990 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:37.412198 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19ee0e-afa5-46d4-8806-b0ba1dfcd990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:37.423552 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19ee0e-afa5-46d4-8806-b0ba1dfcd990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:37.424997 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19ee0e-afa5-46d4-8806-b0ba1dfcd990 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:37.440569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-8e19ee0e-afa5-46d4-8806-b0ba1dfcd990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:37.440569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:37.440569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:37.440569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:37.440569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:37.440569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:37.440569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:37.440569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:37.440569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:37.440569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:37.440569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:37.440569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:37.440569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:37.440569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:37.440569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:37.440569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:37.441785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:37.441785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:37.441785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:37.441785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:37.441785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:37.441785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:37.441785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:37.441785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:37.441785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:37.441785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:37.441785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:37.441785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:37.441785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:37.441785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:37.441785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:37.441785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:37.441785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:37.442915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:37.442915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:37.442915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:37.443473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-6f9678a6-f33c-4758-a487-0e968dd64b5c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 10:18:37.443473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:37.443473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:37.443473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:37.443473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:37.443473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:37.443473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:37.443473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:37.443473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 10:18:37.443473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:37.443473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:37.443473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:37.443473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:37.443473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:37.443473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:37.443473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:37.443473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:37.444645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:37.444645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:37.444645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:37.444645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:37.444645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:37.444645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:37.444645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:37.444645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:37.444645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:37.444645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:37.444645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:37.444645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:37.444645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:37.444645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:37.444645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:37.444645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:37.445758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:37.445758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:37.445758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:37.445758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:37.445758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:37.445758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:37.445758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:37.445758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:37.445758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:37.445758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:37.445758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:37.445758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:37.445758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 10:18:37.445758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:37.455298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 425/844] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:18:37.455298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19ee0e-afa5-46d4-8806-b0ba1dfcd990 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:37.456324 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 420/845] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:18:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:37.459055 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9678a6-f33c-4758-a487-0e968dd64b5c None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:37.537636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:37.597193 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9678a6-f33c-4758-a487-0e968dd64b5c None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:37.631194 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19ee0e-afa5-46d4-8806-b0ba1dfcd990 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:37.654091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedab63-d583-4220-ac07-5b0b1b43c36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:37.654931 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedab63-d583-4220-ac07-5b0b1b43c36b None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:37.655361 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedab63-d583-4220-ac07-5b0b1b43c36b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:37.655793 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedab63-d583-4220-ac07-5b0b1b43c36b None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:37.656569 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedab63-d583-4220-ac07-5b0b1b43c36b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:37.657167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedab63-d583-4220-ac07-5b0b1b43c36b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:37.658770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedab63-d583-4220-ac07-5b0b1b43c36b None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:37.668702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 421/846] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:18:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:37.670388 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedab63-d583-4220-ac07-5b0b1b43c36b None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:37.673308 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ac4ad-ce32-4b63-b98f-7be52556cb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:37.673996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ac4ad-ce32-4b63-b98f-7be52556cb30 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:37.674348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ac4ad-ce32-4b63-b98f-7be52556cb30 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:37.674708 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ac4ad-ce32-4b63-b98f-7be52556cb30 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:37.676107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ac4ad-ce32-4b63-b98f-7be52556cb30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:37.676107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ac4ad-ce32-4b63-b98f-7be52556cb30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:37.677291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ac4ad-ce32-4b63-b98f-7be52556cb30 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:37.695974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 426/847] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:37.697781 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ac4ad-ce32-4b63-b98f-7be52556cb30 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:37.736905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:37.792729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedab63-d583-4220-ac07-5b0b1b43c36b None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:37.803457 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ac4ad-ce32-4b63-b98f-7be52556cb30 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:37.805908 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:37.840657 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549fd7f-3d90-4adf-97fc-463fdb8c683f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:37.841483 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549fd7f-3d90-4adf-97fc-463fdb8c683f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:37.841839 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549fd7f-3d90-4adf-97fc-463fdb8c683f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:37.842165 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549fd7f-3d90-4adf-97fc-463fdb8c683f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:37.844745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10683ad-4741-4df2-932a-bc0b49b68d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:37.846219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10683ad-4741-4df2-932a-bc0b49b68d5f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:37.846700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10683ad-4741-4df2-932a-bc0b49b68d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:37.847155 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10683ad-4741-4df2-932a-bc0b49b68d5f None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:37.848020 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10683ad-4741-4df2-932a-bc0b49b68d5f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:37.848520 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10683ad-4741-4df2-932a-bc0b49b68d5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:37.850028 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549fd7f-3d90-4adf-97fc-463fdb8c683f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:37.850321 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10683ad-4741-4df2-932a-bc0b49b68d5f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:37.851758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549fd7f-3d90-4adf-97fc-463fdb8c683f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:37.863640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-9549fd7f-3d90-4adf-97fc-463fdb8c683f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:37.863640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:37.863640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:37.863640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:37.863640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:37.863640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:37.863640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:37.863640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:37.863640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:37.863640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:37.863640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:37.863640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:37.863640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:37.863640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:37.863640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:37.864641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:37.864641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:37.864641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:37.864641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:37.864641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:37.864641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:37.864641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:37.864641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:37.864641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:37.864641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:37.864641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:37.864641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:37.864641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:37.864641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:37.864641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:37.864641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:37.864641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:37.865883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:37.865883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:37.865883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:37.865883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:37.865883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:37.865883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:37.865883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:37.865883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:37.866816 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 422/849] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:18:37 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:37.870086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 427/849] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:18:37.871272 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10683ad-4741-4df2-932a-bc0b49b68d5f None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:37.990981 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10683ad-4741-4df2-932a-bc0b49b68d5f None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:38.006934 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:38.030923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f713f5-445f-40b3-b5df-c742cf056fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:38.031558 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f713f5-445f-40b3-b5df-c742cf056fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:38.031840 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f713f5-445f-40b3-b5df-c742cf056fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:38.032116 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f713f5-445f-40b3-b5df-c742cf056fa8 None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:38.032827 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f713f5-445f-40b3-b5df-c742cf056fa8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:38.033167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f713f5-445f-40b3-b5df-c742cf056fa8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:38.034710 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f713f5-445f-40b3-b5df-c742cf056fa8 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:38.049220 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 423/850] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:38.055588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549fd7f-3d90-4adf-97fc-463fdb8c683f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:38.071885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26ea74-004f-4215-93e5-310aa7765915 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:38.071885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26ea74-004f-4215-93e5-310aa7765915 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:38.071885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26ea74-004f-4215-93e5-310aa7765915 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:38.073229 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 424/851] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:38.078655 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce28fd0-1b38-4e01-bbc4-7d4c5513d1e1 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:38.079048 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce28fd0-1b38-4e01-bbc4-7d4c5513d1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:38.079223 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce28fd0-1b38-4e01-bbc4-7d4c5513d1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:38.098294 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-5ce28fd0-1b38-4e01-bbc4-7d4c5513d1e1 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:38.099405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5ce28fd0-1b38-4e01-bbc4-7d4c5513d1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:38.183200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 425/852] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:18:38.196620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549fd7f-3d90-4adf-97fc-463fdb8c683f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:38.235489 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925e8d9-1dc1-4d1b-a8ec-5eb82e920071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:38.236480 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925e8d9-1dc1-4d1b-a8ec-5eb82e920071 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:38.237006 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925e8d9-1dc1-4d1b-a8ec-5eb82e920071 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:38.237516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925e8d9-1dc1-4d1b-a8ec-5eb82e920071 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:38.238487 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925e8d9-1dc1-4d1b-a8ec-5eb82e920071 None admin] RBAC: Adding query filter params (role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=17a8bc6c506c45be897bda23eaab485a) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:38.238998 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925e8d9-1dc1-4d1b-a8ec-5eb82e920071 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:38.240598 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925e8d9-1dc1-4d1b-a8ec-5eb82e920071 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:38.248110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 428/853] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:18:38 2022] GET /identity/v3/role_assignments?user.id=17a8bc6c506c45be897bda23eaab485a&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:38.308150 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e4722-1d28-4d29-b46d-31caec8def4d None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:38.308507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e4722-1d28-4d29-b46d-31caec8def4d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:38.308701 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e4722-1d28-4d29-b46d-31caec8def4d None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:38.310280 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 426/854] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:38.317136 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916c34a-268d-4bd7-b22c-015d2e48015b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:38.317136 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916c34a-268d-4bd7-b22c-015d2e48015b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:38.317136 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916c34a-268d-4bd7-b22c-015d2e48015b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:38.336906 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8916c34a-268d-4bd7-b22c-015d2e48015b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:38.338341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8916c34a-268d-4bd7-b22c-015d2e48015b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:38.364819 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:38.414378 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 429/855] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:38.463111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e4722-1d28-4d29-b46d-31caec8def4d None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:38.567502 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e4722-1d28-4d29-b46d-31caec8def4d None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:38.569969 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:38.603172 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbdefe7-ca91-40dd-90ba-c69599d163d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:38.603818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdefe7-ca91-40dd-90ba-c69599d163d4 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:38.604099 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdefe7-ca91-40dd-90ba-c69599d163d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:38.604374 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdefe7-ca91-40dd-90ba-c69599d163d4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:38.605009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbdefe7-ca91-40dd-90ba-c69599d163d4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:38.606860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbdefe7-ca91-40dd-90ba-c69599d163d4 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:38.616496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-dfbdefe7-ca91-40dd-90ba-c69599d163d4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 10:18:38.616496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:38.616496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:38.616496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:38.616496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:38.616496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:38.616496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:38.616496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:38.616496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 10:18:38.616496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:38.616496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:38.616496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:38.616496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:38.616496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:38.616496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:38.616496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:38.617475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:38.617475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:38.617475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:38.617475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:38.617475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:38.617475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:38.617475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:38.617475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:38.617475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:38.617475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:38.617475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:38.617475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:38.617475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:38.617475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:38.617475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:38.617475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:38.617475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:38.618422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:38.618422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:38.618422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:38.618422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:38.618422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:38.618422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:38.618422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:38.618422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:38.618422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:38.618422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:38.618422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:38.618422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:38.618422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 10:18:38.618422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:38.618422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 427/856] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Sep 19 10:18:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:38.624319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916c34a-268d-4bd7-b22c-015d2e48015b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:38.684380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbdefe7-ca91-40dd-90ba-c69599d163d4 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:38.733364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916c34a-268d-4bd7-b22c-015d2e48015b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:38.774191 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406c264-3999-46fe-882a-06b8d9d6c408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:38.775151 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406c264-3999-46fe-882a-06b8d9d6c408 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:38.775897 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406c264-3999-46fe-882a-06b8d9d6c408 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:38.776393 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406c264-3999-46fe-882a-06b8d9d6c408 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:38.777906 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406c264-3999-46fe-882a-06b8d9d6c408 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:38.778217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406c264-3999-46fe-882a-06b8d9d6c408 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:38.779586 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406c264-3999-46fe-882a-06b8d9d6c408 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:38.789113 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 430/857] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 19 10:18:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:38.790107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406c264-3999-46fe-882a-06b8d9d6c408 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:38.816052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbdefe7-ca91-40dd-90ba-c69599d163d4 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:38.825916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:38.862201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05deccfe-f3b9-4ca9-8eb0-283cfe964b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:38.862860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05deccfe-f3b9-4ca9-8eb0-283cfe964b59 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:38.863136 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05deccfe-f3b9-4ca9-8eb0-283cfe964b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:38.863407 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05deccfe-f3b9-4ca9-8eb0-283cfe964b59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:38.874590 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05deccfe-f3b9-4ca9-8eb0-283cfe964b59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:38.880228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-05deccfe-f3b9-4ca9-8eb0-283cfe964b59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:38.880228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:38.880228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:38.880228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:38.880228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:38.880228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:38.880228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:38.880228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:38.880228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:38.880228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:38.880228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:38.880228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:38.880228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:38.880228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:38.880228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:38.880228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:38.881200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:38.881200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:38.881200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:38.881200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:38.881200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:38.881200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:38.881200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:38.881200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:38.881200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:38.881200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:38.881200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:38.881200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:38.881200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:38.881200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:38.881200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:38.881200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:38.882074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:38.882074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:38.882074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:38.882074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:38.882074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:38.882074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:38.882074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:38.882074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:38.882074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:38.882074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:38.882074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:18:38.882074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:38.882074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 428/858] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:18:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:18:38.887076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05deccfe-f3b9-4ca9-8eb0-283cfe964b59 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:38.926488 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:38.928325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406c264-3999-46fe-882a-06b8d9d6c408 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:38.968124 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99d5c1-a81f-4e2b-b18d-5d5b045dadae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:38.968737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99d5c1-a81f-4e2b-b18d-5d5b045dadae None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:38.969020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99d5c1-a81f-4e2b-b18d-5d5b045dadae None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:38.969334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99d5c1-a81f-4e2b-b18d-5d5b045dadae None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:38.970239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99d5c1-a81f-4e2b-b18d-5d5b045dadae None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:38.970551 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99d5c1-a81f-4e2b-b18d-5d5b045dadae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:38.972054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99d5c1-a81f-4e2b-b18d-5d5b045dadae None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:38.977104 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 431/859] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 19 10:18:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:39.014676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05deccfe-f3b9-4ca9-8eb0-283cfe964b59 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:39.056832 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2cee3-d17c-4740-9aeb-b033991ba5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:39.057795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2cee3-d17c-4740-9aeb-b033991ba5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:39.058222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2cee3-d17c-4740-9aeb-b033991ba5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:39.058720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2cee3-d17c-4740-9aeb-b033991ba5d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:39.059571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2cee3-d17c-4740-9aeb-b033991ba5d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:39.060028 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2cee3-d17c-4740-9aeb-b033991ba5d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:39.061674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2cee3-d17c-4740-9aeb-b033991ba5d4 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:39.072271 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 429/860] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:18:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:39.082311 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99d5c1-a81f-4e2b-b18d-5d5b045dadae None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:39.177956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:39.194167 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99d5c1-a81f-4e2b-b18d-5d5b045dadae None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:39.230345 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5854b-1bef-4178-bcc3-cf55c4c24a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:39.231417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5854b-1bef-4178-bcc3-cf55c4c24a7c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:39.232014 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5854b-1bef-4178-bcc3-cf55c4c24a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:39.232438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5854b-1bef-4178-bcc3-cf55c4c24a7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:39.243242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5854b-1bef-4178-bcc3-cf55c4c24a7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:39.245519 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5854b-1bef-4178-bcc3-cf55c4c24a7c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:39.261275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-19c5854b-1bef-4178-bcc3-cf55c4c24a7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:39.261275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:39.261275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:39.261275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:39.261275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:39.261275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:39.261275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:39.261275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:39.261275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:39.261275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:39.261275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:39.261275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:39.261275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:39.261275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:39.261275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:39.261275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:39.262478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:39.262478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:39.262478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:39.262478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:39.262478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:39.262478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:39.262478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:39.262478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:39.262478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:39.262478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:39.262478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:39.262478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:39.262478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:39.262478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:39.262478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:39.262478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:39.262478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:39.263609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:39.263609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:39.263609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:39.265295 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 432/861] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:39.271175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2cee3-d17c-4740-9aeb-b033991ba5d4 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:39.368446 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:39.379089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2cee3-d17c-4740-9aeb-b033991ba5d4 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:39.414989 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a4bdf-ad0f-442c-b691-6df4232363c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:39.416002 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a4bdf-ad0f-442c-b691-6df4232363c7 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:39.416443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a4bdf-ad0f-442c-b691-6df4232363c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:39.416861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a4bdf-ad0f-442c-b691-6df4232363c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:39.417723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a4bdf-ad0f-442c-b691-6df4232363c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:39.418266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a4bdf-ad0f-442c-b691-6df4232363c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:39.419878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a4bdf-ad0f-442c-b691-6df4232363c7 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:39.432308 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 430/862] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:39.438019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5854b-1bef-4178-bcc3-cf55c4c24a7c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:39.542777 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5854b-1bef-4178-bcc3-cf55c4c24a7c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:39.578560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80965390-5a3c-426f-8006-cbfa3e2877f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:39.579516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80965390-5a3c-426f-8006-cbfa3e2877f7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:39.579942 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80965390-5a3c-426f-8006-cbfa3e2877f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:39.580361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80965390-5a3c-426f-8006-cbfa3e2877f7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:39.588369 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80965390-5a3c-426f-8006-cbfa3e2877f7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:39.590144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80965390-5a3c-426f-8006-cbfa3e2877f7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:39.592029 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:39.601669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-80965390-5a3c-426f-8006-cbfa3e2877f7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 10:18:39.601669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:39.601669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:39.601669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:39.601669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:39.601669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:39.601669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:39.601669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:39.601669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:39.601669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:39.601669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:39.601669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:39.601669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:39.601669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:39.601669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:39.601669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:39.602628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:39.602628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:39.602628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:39.602628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:39.602628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:39.602628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:39.602628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:39.602628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:39.602628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:39.602628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:39.602628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:39.602628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:39.602628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:39.602628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:39.602628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:39.602628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:39.602628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:39.602628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:39.603572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:39.603572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:39.603572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:39.603572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:39.603572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:39.603572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:39.603572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:39.603572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:39.603572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 10:18:39.603572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:39.603572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 433/863] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 19 10:18:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:39.608797 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a4bdf-ad0f-442c-b691-6df4232363c7 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:39.720158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a4bdf-ad0f-442c-b691-6df4232363c7 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:39.758644 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd3ddc-07f9-403f-bd8c-764f994cc935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:39.760222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd3ddc-07f9-403f-bd8c-764f994cc935 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:39.760699 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd3ddc-07f9-403f-bd8c-764f994cc935 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:39.761226 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd3ddc-07f9-403f-bd8c-764f994cc935 None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:39.762135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd3ddc-07f9-403f-bd8c-764f994cc935 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:39.762633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd3ddc-07f9-403f-bd8c-764f994cc935 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:39.764248 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd3ddc-07f9-403f-bd8c-764f994cc935 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:39.782931 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 431/864] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Sep 19 10:18:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:39.790925 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80965390-5a3c-426f-8006-cbfa3e2877f7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:39.895295 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80965390-5a3c-426f-8006-cbfa3e2877f7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:39.930149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05647745-9ff2-4cef-bb1c-912c57daca37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:39.931266 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05647745-9ff2-4cef-bb1c-912c57daca37 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:39.931552 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05647745-9ff2-4cef-bb1c-912c57daca37 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:39.931826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05647745-9ff2-4cef-bb1c-912c57daca37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:40.001483 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:40.006069 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05647745-9ff2-4cef-bb1c-912c57daca37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:40.007332 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05647745-9ff2-4cef-bb1c-912c57daca37 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:40.115342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 434/865] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:18:39 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:18:40.121133 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd3ddc-07f9-403f-bd8c-764f994cc935 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:40.131984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:40.137178 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27842f9-5d6c-419f-90e2-161a2345e46a None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:40.137468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27842f9-5d6c-419f-90e2-161a2345e46a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:40.137734 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27842f9-5d6c-419f-90e2-161a2345e46a None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:40.139290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 435/866] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:18:40.145373 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51056d9-e9e1-438d-a7cd-5b35819f7596 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:40.145773 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51056d9-e9e1-438d-a7cd-5b35819f7596 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:40.146053 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51056d9-e9e1-438d-a7cd-5b35819f7596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:40.166412 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-d51056d9-e9e1-438d-a7cd-5b35819f7596 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:40.167600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d51056d9-e9e1-438d-a7cd-5b35819f7596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:40.244133 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd3ddc-07f9-403f-bd8c-764f994cc935 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:40.249217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 436/867] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:40.289375 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40d134-4ab8-4603-943c-3f3a022f4c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:40.289938 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40d134-4ab8-4603-943c-3f3a022f4c1f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:40.291945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40d134-4ab8-4603-943c-3f3a022f4c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:40.291945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40d134-4ab8-4603-943c-3f3a022f4c1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:40.302994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40d134-4ab8-4603-943c-3f3a022f4c1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:40.304454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40d134-4ab8-4603-943c-3f3a022f4c1f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:40.320437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-4c40d134-4ab8-4603-943c-3f3a022f4c1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:40.320437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:40.320437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:40.320437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:40.320437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:40.320437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:40.320437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:40.320437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:40.320437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:40.320437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:40.320437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:40.320437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:40.320437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:40.320437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:40.320437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:40.320437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:40.322076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:40.322076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:40.322076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:40.322076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:40.322076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:40.322076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:40.322076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:40.322076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:40.322076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:40.322076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:40.322076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:40.322076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:40.322076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:40.322076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:40.322076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:40.322076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:40.322076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:40.322962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:40.322962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:40.322962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:40.322962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 432/868] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:18:40.330163 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51056d9-e9e1-438d-a7cd-5b35819f7596 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:40.359396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:40.435880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51056d9-e9e1-438d-a7cd-5b35819f7596 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:40.472102 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c06787-9763-40b1-a65b-3e5d160c85d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:40.472716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c06787-9763-40b1-a65b-3e5d160c85d2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:40.472995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c06787-9763-40b1-a65b-3e5d160c85d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:40.473301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c06787-9763-40b1-a65b-3e5d160c85d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:40.473953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c06787-9763-40b1-a65b-3e5d160c85d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:40.474260 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c06787-9763-40b1-a65b-3e5d160c85d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:40.475626 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c06787-9763-40b1-a65b-3e5d160c85d2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:40.488076 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 437/869] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:40.494162 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40d134-4ab8-4603-943c-3f3a022f4c1f None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:40.551495 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c06787-9763-40b1-a65b-3e5d160c85d2 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:40.563568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:40.618605 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40d134-4ab8-4603-943c-3f3a022f4c1f None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:40.651778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:40.662998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b1bac-5850-4bb5-9d0b-8744519f31a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:40.662998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b1bac-5850-4bb5-9d0b-8744519f31a2 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:40.662998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b1bac-5850-4bb5-9d0b-8744519f31a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:40.662998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b1bac-5850-4bb5-9d0b-8744519f31a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:40.665133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c06787-9763-40b1-a65b-3e5d160c85d2 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:40.670511 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b1bac-5850-4bb5-9d0b-8744519f31a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:40.671979 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b1bac-5850-4bb5-9d0b-8744519f31a2 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:40.683454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-d59b1bac-5850-4bb5-9d0b-8744519f31a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:40.683454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:40.683454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:40.683454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:40.683454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:40.683454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:40.683454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:40.683454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:40.683454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:40.683454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:40.683454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:40.683454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:40.683454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:40.683454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:40.683454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:40.684343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:40.684343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:40.684343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:40.684343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:40.684343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:40.684343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:40.684343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:40.684343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:40.684343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:40.684343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:40.684343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:40.684343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:40.684343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:40.684343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:40.684343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:40.684343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:40.684343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:40.685283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:40.685283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:40.685283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:40.685283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:40.685283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:40.685283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:40.685283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:40.685283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:40.685283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 433/870] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:40.685777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534b554-a7ad-4d96-b074-d6cbd94cd761 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:40.686163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534b554-a7ad-4d96-b074-d6cbd94cd761 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:40.686260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534b554-a7ad-4d96-b074-d6cbd94cd761 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:40.691449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 434/871] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:40.693130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e2d4b-35cd-438e-97e1-e66a490cabbc None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:40.693412 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e2d4b-35cd-438e-97e1-e66a490cabbc None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:40.693683 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e2d4b-35cd-438e-97e1-e66a490cabbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:40.710336 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4aa762-1dcd-47e1-9bf9-a98e4edfbfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:40.710939 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4aa762-1dcd-47e1-9bf9-a98e4edfbfee None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:40.711212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4aa762-1dcd-47e1-9bf9-a98e4edfbfee None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:40.711484 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4aa762-1dcd-47e1-9bf9-a98e4edfbfee None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:40.712722 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4aa762-1dcd-47e1-9bf9-a98e4edfbfee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:40.712935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-059e2d4b-35cd-438e-97e1-e66a490cabbc None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:40.714096 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4aa762-1dcd-47e1-9bf9-a98e4edfbfee None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:40.714196 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-059e2d4b-35cd-438e-97e1-e66a490cabbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:40.780036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 435/872] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:40.780946 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e2d4b-35cd-438e-97e1-e66a490cabbc None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:40.789943 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 438/873] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 19 10:18:40 2022] POST /identity/v3/services => generated 251 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:18:40.843679 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:40.907433 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e2d4b-35cd-438e-97e1-e66a490cabbc None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:40.954010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fb3ac-c875-4928-b477-c121ea5b8de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:40.954631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fb3ac-c875-4928-b477-c121ea5b8de8 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:40.954910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fb3ac-c875-4928-b477-c121ea5b8de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:40.955180 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fb3ac-c875-4928-b477-c121ea5b8de8 None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:40.955874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fb3ac-c875-4928-b477-c121ea5b8de8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:40.956177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fb3ac-c875-4928-b477-c121ea5b8de8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:40.957737 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fb3ac-c875-4928-b477-c121ea5b8de8 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:40.971779 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 436/874] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:40.978305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4aa762-1dcd-47e1-9bf9-a98e4edfbfee None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:41.078258 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fb3ac-c875-4928-b477-c121ea5b8de8 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:41.093617 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4aa762-1dcd-47e1-9bf9-a98e4edfbfee None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:41.130492 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528b9a2-d27b-438a-9643-f3d6017c3de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:41.131104 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528b9a2-d27b-438a-9643-f3d6017c3de8 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:41.131388 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528b9a2-d27b-438a-9643-f3d6017c3de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:41.131668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528b9a2-d27b-438a-9643-f3d6017c3de8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:41.132534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528b9a2-d27b-438a-9643-f3d6017c3de8 None admin] RBAC: Adding query filter params (role.id=39be941a839d4eb6a2146c561c7a2851, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=4ae237601b4a4aab9e969fedb5ee37d7) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:41.132847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528b9a2-d27b-438a-9643-f3d6017c3de8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:41.135044 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528b9a2-d27b-438a-9643-f3d6017c3de8 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:41.149231 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 439/875] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:18:40 2022] GET /identity/v3/role_assignments?user.id=4ae237601b4a4aab9e969fedb5ee37d7&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=39be941a839d4eb6a2146c561c7a2851 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:41.202038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fb3ac-c875-4928-b477-c121ea5b8de8 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:41.210848 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:41.240429 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6f02c-b58e-499e-8d33-556b75ab4b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:41.241102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6f02c-b58e-499e-8d33-556b75ab4b99 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:41.241391 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6f02c-b58e-499e-8d33-556b75ab4b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:41.241664 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6f02c-b58e-499e-8d33-556b75ab4b99 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:41.242261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6f02c-b58e-499e-8d33-556b75ab4b99 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:41.244202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6f02c-b58e-499e-8d33-556b75ab4b99 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:41.254087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-dea6f02c-b58e-499e-8d33-556b75ab4b99 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 10:18:41.254087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:41.254087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:41.254087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:41.254087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:41.254087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:41.254087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:41.254087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:41.254087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 10:18:41.254087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:41.254087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:41.254087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:41.254087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:41.254087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:41.254087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:41.254087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:41.254087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:41.255164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:41.255164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:41.255164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:41.255164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:41.255164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:41.255164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:41.255164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:41.255164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:41.255164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:41.255164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:41.255164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:41.255164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:41.255164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:41.255164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:41.255164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:41.255164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:41.256630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:41.256630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:41.256630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:41.256630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:41.256630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:41.256630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:41.256630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:41.256630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:41.256630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:41.256630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:41.256630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:41.256630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:41.256630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 10:18:41.256630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:41.256630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 437/876] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 19 10:18:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:41.262569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528b9a2-d27b-438a-9643-f3d6017c3de8 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:41.270061 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4840737-3de4-4a1c-974a-e4b53fb6e057 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:41.270396 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4840737-3de4-4a1c-974a-e4b53fb6e057 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:41.270612 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4840737-3de4-4a1c-974a-e4b53fb6e057 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:41.272229 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 438/877] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:18:41.278744 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa3373-6576-4ee5-a9a3-03869bc35562 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:41.279045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa3373-6576-4ee5-a9a3-03869bc35562 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:41.279303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa3373-6576-4ee5-a9a3-03869bc35562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:41.285587 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:41.304290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-b0fa3373-6576-4ee5-a9a3-03869bc35562 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:41.306571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b0fa3373-6576-4ee5-a9a3-03869bc35562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:41.401179 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 439/878] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:41 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:18:41.403346 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528b9a2-d27b-438a-9643-f3d6017c3de8 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:41.440385 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c8330-d715-4fde-8593-1c2b222d41ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:41.440992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c8330-d715-4fde-8593-1c2b222d41ac None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:41.441301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c8330-d715-4fde-8593-1c2b222d41ac None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:41.441572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c8330-d715-4fde-8593-1c2b222d41ac None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:41.442203 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c8330-d715-4fde-8593-1c2b222d41ac None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:41.442502 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c8330-d715-4fde-8593-1c2b222d41ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:41.443857 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c8330-d715-4fde-8593-1c2b222d41ac None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:41.449606 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 440/879] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 19 10:18:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:41.455182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa3373-6576-4ee5-a9a3-03869bc35562 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:41.564825 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:41.568772 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa3373-6576-4ee5-a9a3-03869bc35562 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:41.607444 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33c583-3c3b-44b2-a0d2-6ab97cf87408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:41.608428 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33c583-3c3b-44b2-a0d2-6ab97cf87408 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:41.608908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33c583-3c3b-44b2-a0d2-6ab97cf87408 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:41.609405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33c583-3c3b-44b2-a0d2-6ab97cf87408 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:41.610352 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33c583-3c3b-44b2-a0d2-6ab97cf87408 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:41.610885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33c583-3c3b-44b2-a0d2-6ab97cf87408 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:41.612467 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33c583-3c3b-44b2-a0d2-6ab97cf87408 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:41.618058 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 440/880] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 19 10:18:41 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:41.674832 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c8330-d715-4fde-8593-1c2b222d41ac None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:41.769215 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:41.797097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c8330-d715-4fde-8593-1c2b222d41ac None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:41.840400 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0333bde-d91a-40d6-a300-2e1d26a85aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:41.841013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0333bde-d91a-40d6-a300-2e1d26a85aaf None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:41.841321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0333bde-d91a-40d6-a300-2e1d26a85aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:41.841595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0333bde-d91a-40d6-a300-2e1d26a85aaf None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:41.842251 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0333bde-d91a-40d6-a300-2e1d26a85aaf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:41.843680 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0333bde-d91a-40d6-a300-2e1d26a85aaf None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:41.908691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 441/881] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 19 10:18:41 2022] POST /identity/v3/services => generated 269 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:18:43.104491 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777d46c-d8be-4a25-8fb9-7cf77d548abc None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:43.105004 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777d46c-d8be-4a25-8fb9-7cf77d548abc None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:43.105004 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777d46c-d8be-4a25-8fb9-7cf77d548abc None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:43.106625 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 441/882] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:43.112178 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812f41b-cd2c-460b-a090-3764c9004b85 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:43.112574 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812f41b-cd2c-460b-a090-3764c9004b85 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:43.112759 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812f41b-cd2c-460b-a090-3764c9004b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:43.136060 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4812f41b-cd2c-460b-a090-3764c9004b85 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:43.137267 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4812f41b-cd2c-460b-a090-3764c9004b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:43.212670 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 442/883] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:43.489488 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777d46c-d8be-4a25-8fb9-7cf77d548abc None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:43.528554 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a744f9-1d5c-4b14-83a7-c8aaedf8fd34 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:43.528910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a744f9-1d5c-4b14-83a7-c8aaedf8fd34 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:43.529129 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a744f9-1d5c-4b14-83a7-c8aaedf8fd34 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:43.530699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 443/884] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:18:43.535932 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bead3-4ad7-4af0-b8ff-7e83bda97b89 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:43.536219 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bead3-4ad7-4af0-b8ff-7e83bda97b89 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:43.536493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bead3-4ad7-4af0-b8ff-7e83bda97b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:43.558552 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b11bead3-4ad7-4af0-b8ff-7e83bda97b89 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:43.558852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b11bead3-4ad7-4af0-b8ff-7e83bda97b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:43.601604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777d46c-d8be-4a25-8fb9-7cf77d548abc None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:43.633860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:43.645827 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec513dbe-ff03-4801-8a92-6d7f6806dc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:43.646289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 444/885] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:18:43.646431 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec513dbe-ff03-4801-8a92-6d7f6806dc92 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:43.646811 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec513dbe-ff03-4801-8a92-6d7f6806dc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:43.647043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec513dbe-ff03-4801-8a92-6d7f6806dc92 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:43.647952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec513dbe-ff03-4801-8a92-6d7f6806dc92 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:43.653166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec513dbe-ff03-4801-8a92-6d7f6806dc92 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:43.662789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-ec513dbe-ff03-4801-8a92-6d7f6806dc92 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 10:18:43.662789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:43.662789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:43.662789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:43.662789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:43.662789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:43.662789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:43.662789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:43.662789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 10:18:43.662789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:43.662789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:43.662789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:43.662789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:43.662789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:43.662789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:43.662789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:43.662789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:43.664159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:43.664159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:43.664159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:43.664159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:43.664159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:43.664159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:43.664159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:43.664159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:43.664159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:43.664159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:43.664159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:43.664159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:43.664159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:43.664159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:43.664159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:43.664159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:43.666147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:43.666147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:43.666147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:43.666147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:43.666147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:43.666147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:43.666147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:43.666147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:43.666147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:43.666147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:43.666147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:43.666147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:43.666147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 10:18:43.666147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:43.669907 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 442/886] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:18:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:43.675518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bead3-4ad7-4af0-b8ff-7e83bda97b89 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:43.785786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bead3-4ad7-4af0-b8ff-7e83bda97b89 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:43.827838 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d373a-5e46-4485-8904-6e5ccd849d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:43.828456 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d373a-5e46-4485-8904-6e5ccd849d19 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:43.828733 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d373a-5e46-4485-8904-6e5ccd849d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:43.829007 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d373a-5e46-4485-8904-6e5ccd849d19 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:43.829676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d373a-5e46-4485-8904-6e5ccd849d19 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:43.829979 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d373a-5e46-4485-8904-6e5ccd849d19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:43.831335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d373a-5e46-4485-8904-6e5ccd849d19 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:43.836383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 445/887] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:18:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:43.842754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec513dbe-ff03-4801-8a92-6d7f6806dc92 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:43.872678 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:43.900000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79dda2-c6e4-4775-a4b6-e5ffc7400faf None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:43.900322 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79dda2-c6e4-4775-a4b6-e5ffc7400faf None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:43.900556 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79dda2-c6e4-4775-a4b6-e5ffc7400faf None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:43.902491 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 446/888] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:43.908744 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666ec29-69f9-4b9e-9c60-30854e717d5f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:43.909024 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666ec29-69f9-4b9e-9c60-30854e717d5f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:43.909340 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666ec29-69f9-4b9e-9c60-30854e717d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:43.930331 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4666ec29-69f9-4b9e-9c60-30854e717d5f None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:43.931718 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4666ec29-69f9-4b9e-9c60-30854e717d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:43.954572 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec513dbe-ff03-4801-8a92-6d7f6806dc92 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:43.995835 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:44.001721 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468e996-5dbd-4eda-96dc-2f6b9022aa22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:44.006206 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468e996-5dbd-4eda-96dc-2f6b9022aa22 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:44.006206 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468e996-5dbd-4eda-96dc-2f6b9022aa22 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:44.006206 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468e996-5dbd-4eda-96dc-2f6b9022aa22 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:44.006206 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468e996-5dbd-4eda-96dc-2f6b9022aa22 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:44.006206 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468e996-5dbd-4eda-96dc-2f6b9022aa22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:44.006206 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468e996-5dbd-4eda-96dc-2f6b9022aa22 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:44.010688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 443/889] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:18:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:44.010688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468e996-5dbd-4eda-96dc-2f6b9022aa22 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:44.017192 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 447/890] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:18:44.022000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666ec29-69f9-4b9e-9c60-30854e717d5f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:44.112065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:44.121742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468e996-5dbd-4eda-96dc-2f6b9022aa22 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:44.139425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666ec29-69f9-4b9e-9c60-30854e717d5f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:44.159005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e922c-10d2-4a9f-8b69-2c1788e522a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:44.159635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e922c-10d2-4a9f-8b69-2c1788e522a0 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:44.159914 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e922c-10d2-4a9f-8b69-2c1788e522a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:44.160190 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e922c-10d2-4a9f-8b69-2c1788e522a0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:44.171095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e922c-10d2-4a9f-8b69-2c1788e522a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:44.174596 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33dad2-c861-4dab-8daf-5b8e62580027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:44.175201 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33dad2-c861-4dab-8daf-5b8e62580027 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:44.175482 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33dad2-c861-4dab-8daf-5b8e62580027 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:44.175759 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33dad2-c861-4dab-8daf-5b8e62580027 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:44.176254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-7a2e922c-10d2-4a9f-8b69-2c1788e522a0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 10:18:44.176254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:44.176254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:44.176254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:44.176254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:44.176254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:44.176254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:44.176254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:44.176254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:44.176254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:44.176254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:44.176254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:44.176254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:44.176254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:44.176254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:44.176254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:44.177200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:44.177200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:44.177200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:44.177200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:44.177200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:44.177200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:44.177200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:44.177200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:44.177200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:44.177200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:44.177200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:44.177200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:44.177200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:44.177200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:44.177200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:44.177200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:44.183903 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:44.183903 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:44.183903 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:44.183903 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:44.183903 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:44.183903 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:44.183903 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:44.183903 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:44.183903 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:44.183903 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:44.183903 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 10:18:44.183903 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:44.184580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33dad2-c861-4dab-8daf-5b8e62580027 None admin] RBAC: Adding query filter params (interface=public, service_id=90ffef1f3fbe43cb9738896fb40911d8, region_id=RegionOne) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:44.184580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33dad2-c861-4dab-8daf-5b8e62580027 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:44.184719 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 444/891] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Sep 19 10:18:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:44.184787 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33dad2-c861-4dab-8daf-5b8e62580027 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:44.184854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828d763-7796-41f5-9593-5cdd397cc78f None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:44.184854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828d763-7796-41f5-9593-5cdd397cc78f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:44.184854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828d763-7796-41f5-9593-5cdd397cc78f None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:44.184854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 445/892] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:44.184854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828d763-7796-41f5-9593-5cdd397cc78f None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:44.189726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 448/893] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Sep 19 10:18:44 2022] GET /identity/v3/endpoints?service_id=90ffef1f3fbe43cb9738896fb40911d8&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:44.194945 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d4a78-a8b9-4bc8-9e52-4206188bc32a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:44.195236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d4a78-a8b9-4bc8-9e52-4206188bc32a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:44.195517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d4a78-a8b9-4bc8-9e52-4206188bc32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:44.214875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f84d4a78-a8b9-4bc8-9e52-4206188bc32a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:44.216080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f84d4a78-a8b9-4bc8-9e52-4206188bc32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:44.261578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:44.285098 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 449/894] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:44.285098 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d4a78-a8b9-4bc8-9e52-4206188bc32a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:44.310020 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828d763-7796-41f5-9593-5cdd397cc78f None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:44.343441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fa26b-4407-43f9-90ba-1794b1b341dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:44.344040 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fa26b-4407-43f9-90ba-1794b1b341dc None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:44.344313 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fa26b-4407-43f9-90ba-1794b1b341dc None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:44.344591 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fa26b-4407-43f9-90ba-1794b1b341dc None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:44.345273 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fa26b-4407-43f9-90ba-1794b1b341dc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:44.345571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fa26b-4407-43f9-90ba-1794b1b341dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:44.346929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fa26b-4407-43f9-90ba-1794b1b341dc None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:44.355388 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 446/895] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Sep 19 10:18:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:44.356176 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fa26b-4407-43f9-90ba-1794b1b341dc None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:44.374183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:44.396876 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d4a78-a8b9-4bc8-9e52-4206188bc32a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:44.436312 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76a0ce-9024-4fdb-a167-e628e106711e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:44.437361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76a0ce-9024-4fdb-a167-e628e106711e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:44.437704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76a0ce-9024-4fdb-a167-e628e106711e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:44.437981 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76a0ce-9024-4fdb-a167-e628e106711e None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:44.438951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76a0ce-9024-4fdb-a167-e628e106711e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:44.440619 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76a0ce-9024-4fdb-a167-e628e106711e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:44.451975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 450/896] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Sep 19 10:18:44 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:18:44.453005 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76a0ce-9024-4fdb-a167-e628e106711e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:44.474662 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fa26b-4407-43f9-90ba-1794b1b341dc None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:44.512228 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:44.516995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e3f9d-c2ec-4046-a788-9a44114aa616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:44.517970 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e3f9d-c2ec-4046-a788-9a44114aa616 None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:44.518427 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e3f9d-c2ec-4046-a788-9a44114aa616 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:44.518953 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e3f9d-c2ec-4046-a788-9a44114aa616 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:44.519892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e3f9d-c2ec-4046-a788-9a44114aa616 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:44.521562 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e3f9d-c2ec-4046-a788-9a44114aa616 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:44.574048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76a0ce-9024-4fdb-a167-e628e106711e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:44.606587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 447/897] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 19 10:18:44 2022] POST /identity/v3/services => generated 241 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:18:44.607835 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e3f9d-c2ec-4046-a788-9a44114aa616 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:44.625710 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3a750-8be6-44d5-bc29-6b3b5477061a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:44.627421 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3a750-8be6-44d5-bc29-6b3b5477061a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:44.627703 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3a750-8be6-44d5-bc29-6b3b5477061a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:44.627980 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3a750-8be6-44d5-bc29-6b3b5477061a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:44.643800 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3a750-8be6-44d5-bc29-6b3b5477061a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:44.645434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3a750-8be6-44d5-bc29-6b3b5477061a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:44.668476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-c5a3a750-8be6-44d5-bc29-6b3b5477061a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:44.668476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:44.668476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:44.668476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:44.668476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:44.668476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:44.668476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:44.668476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:44.668476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:44.668476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:44.668476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:44.668476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:44.668476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:44.668476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:44.668476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:44.668476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:44.669578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:44.669578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:44.669578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:44.669578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:44.669578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:44.669578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:44.669578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:44.669578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:44.669578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:44.669578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:44.669578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:44.669578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:44.669578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:44.669578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:44.669578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:44.669578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:44.669578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:44.670481 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:44.670481 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:44.670481 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:44.670481 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 451/898] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:44.676503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3a750-8be6-44d5-bc29-6b3b5477061a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:44.711342 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:44.748060 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e3f9d-c2ec-4046-a788-9a44114aa616 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:44.791441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd39349-6866-4099-9284-9c31128798fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:44.792473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd39349-6866-4099-9284-9c31128798fb None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:44.793050 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd39349-6866-4099-9284-9c31128798fb None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:44.793606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd39349-6866-4099-9284-9c31128798fb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:44.794129 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:44.795594 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd39349-6866-4099-9284-9c31128798fb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:44.797278 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd39349-6866-4099-9284-9c31128798fb None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:44.804201 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3a750-8be6-44d5-bc29-6b3b5477061a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:44.808238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-8fd39349-6866-4099-9284-9c31128798fb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 10:18:44.808238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:44.808238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:44.808238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:44.808238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:44.808238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:44.808238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:44.808238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:44.808238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 10:18:44.808238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:44.808238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:44.808238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:44.808238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:44.808238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:44.808238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:44.808238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:44.809212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:44.809212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:44.809212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:44.809212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:44.809212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:44.809212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:44.809212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:44.809212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:44.809212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:44.809212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:44.809212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:44.809212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:44.809212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:44.809212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:44.809212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:44.809212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:44.809212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:44.810102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:44.810102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:44.810102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:44.810102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:44.810102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:44.810102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:44.810102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:44.810102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:44.810102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:44.810102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:44.810102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:44.810102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:44.810102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 10:18:44.810102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:44.812832 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 448/899] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Sep 19 10:18:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:44.820202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd39349-6866-4099-9284-9c31128798fb None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:44.856503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d55a9b-d4d8-4c8c-8b79-5b706b00d638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:44.857145 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d55a9b-d4d8-4c8c-8b79-5b706b00d638 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:44.857439 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d55a9b-d4d8-4c8c-8b79-5b706b00d638 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:44.857756 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d55a9b-d4d8-4c8c-8b79-5b706b00d638 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:44.858719 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d55a9b-d4d8-4c8c-8b79-5b706b00d638 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:44.859910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d55a9b-d4d8-4c8c-8b79-5b706b00d638 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:44.861755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d55a9b-d4d8-4c8c-8b79-5b706b00d638 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:44.879647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 452/900] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:44.886223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d55a9b-d4d8-4c8c-8b79-5b706b00d638 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:44.939241 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd39349-6866-4099-9284-9c31128798fb None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:44.974460 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4a796-0683-4e56-b7fc-281cd84e9109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:44.975347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4a796-0683-4e56-b7fc-281cd84e9109 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:44.975794 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4a796-0683-4e56-b7fc-281cd84e9109 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:44.976223 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4a796-0683-4e56-b7fc-281cd84e9109 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:44.977001 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4a796-0683-4e56-b7fc-281cd84e9109 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:44.978073 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4a796-0683-4e56-b7fc-281cd84e9109 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:44.979698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4a796-0683-4e56-b7fc-281cd84e9109 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:44.985033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 449/901] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 19 10:18:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:44.991397 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4a796-0683-4e56-b7fc-281cd84e9109 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:44.999906 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d55a9b-d4d8-4c8c-8b79-5b706b00d638 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:45.005420 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:45.047055 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebac782-7fd1-4f28-a472-2b3deca9dbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:45.048074 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebac782-7fd1-4f28-a472-2b3deca9dbc1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:45.048362 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebac782-7fd1-4f28-a472-2b3deca9dbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:45.048843 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebac782-7fd1-4f28-a472-2b3deca9dbc1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:45.058022 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebac782-7fd1-4f28-a472-2b3deca9dbc1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:45.059675 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebac782-7fd1-4f28-a472-2b3deca9dbc1 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:45.074874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-bebac782-7fd1-4f28-a472-2b3deca9dbc1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 10:18:45.074874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:45.074874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:45.074874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:45.074874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:45.074874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:45.074874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:45.074874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:45.074874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:45.074874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:45.074874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:45.074874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:45.074874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:45.074874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:45.074874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:45.074874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:45.075954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:45.075954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:45.075954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:45.075954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:45.075954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:45.075954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:45.075954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:45.075954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:45.075954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:45.075954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:45.075954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:45.075954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:45.075954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:45.075954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:45.075954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:45.075954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:45.075954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:45.075954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:45.077612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:45.077612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:45.077612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:45.077612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:45.077612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:45.077612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:45.077612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:45.077612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:45.077612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 10:18:45.077612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:45.077612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 453/902] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 19 10:18:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:45.081517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebac782-7fd1-4f28-a472-2b3deca9dbc1 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:45.105173 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:45.127553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4a796-0683-4e56-b7fc-281cd84e9109 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:45.172718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd9934-6d07-4f64-a538-9ca772623362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:45.173726 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd9934-6d07-4f64-a538-9ca772623362 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:45.174674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd9934-6d07-4f64-a538-9ca772623362 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:45.175171 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd9934-6d07-4f64-a538-9ca772623362 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:45.176378 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd9934-6d07-4f64-a538-9ca772623362 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:45.176912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd9934-6d07-4f64-a538-9ca772623362 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:45.179184 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd9934-6d07-4f64-a538-9ca772623362 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:45.191557 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 450/903] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 19 10:18:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:45.193481 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd9934-6d07-4f64-a538-9ca772623362 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:45.207806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebac782-7fd1-4f28-a472-2b3deca9dbc1 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:45.260032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15510d-68dc-4af4-9bf7-fd45a17b2a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:45.261630 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15510d-68dc-4af4-9bf7-fd45a17b2a17 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:45.262141 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15510d-68dc-4af4-9bf7-fd45a17b2a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:45.262640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15510d-68dc-4af4-9bf7-fd45a17b2a17 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:45.263586 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15510d-68dc-4af4-9bf7-fd45a17b2a17 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:45.264010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:45.264346 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15510d-68dc-4af4-9bf7-fd45a17b2a17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:45.266697 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15510d-68dc-4af4-9bf7-fd45a17b2a17 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:45.290963 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 454/904] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Sep 19 10:18:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:45.297262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15510d-68dc-4af4-9bf7-fd45a17b2a17 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:45.303878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd9934-6d07-4f64-a538-9ca772623362 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:45.342561 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a227b-8acd-409c-9cbf-676106eec5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:45.343290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a227b-8acd-409c-9cbf-676106eec5ab None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:45.343512 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a227b-8acd-409c-9cbf-676106eec5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:45.343788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a227b-8acd-409c-9cbf-676106eec5ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:45.344550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a227b-8acd-409c-9cbf-676106eec5ab None admin] RBAC: Adding query filter params (interface=public, service_id=734970c034ac4710a8e9a4e89ad6a647, region_id=RegionOne) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:45.344866 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a227b-8acd-409c-9cbf-676106eec5ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:45.346349 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a227b-8acd-409c-9cbf-676106eec5ab None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:45.361920 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 451/905] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Sep 19 10:18:45 2022] GET /identity/v3/endpoints?service_id=734970c034ac4710a8e9a4e89ad6a647&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:45.364071 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a227b-8acd-409c-9cbf-676106eec5ab None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:45.392676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:45.426557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15510d-68dc-4af4-9bf7-fd45a17b2a17 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:45.466587 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46c352-baca-4d81-8e0e-b1399f0bfcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:45.467195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46c352-baca-4d81-8e0e-b1399f0bfcc0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:45.467477 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46c352-baca-4d81-8e0e-b1399f0bfcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:45.467755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46c352-baca-4d81-8e0e-b1399f0bfcc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:45.487587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a227b-8acd-409c-9cbf-676106eec5ab None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:45.517623 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:45.525784 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad6e59-e813-4b55-94fb-f55cbdc84fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:45.530955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad6e59-e813-4b55-94fb-f55cbdc84fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:45.530955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad6e59-e813-4b55-94fb-f55cbdc84fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:45.530955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad6e59-e813-4b55-94fb-f55cbdc84fc5 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:45.530955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad6e59-e813-4b55-94fb-f55cbdc84fc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:45.530955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad6e59-e813-4b55-94fb-f55cbdc84fc5 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:45.544077 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 452/906] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Sep 19 10:18:45 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:18:45.548714 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c46c352-baca-4d81-8e0e-b1399f0bfcc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:45.550265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c46c352-baca-4d81-8e0e-b1399f0bfcc0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:45.629597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:45.665387 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 455/907] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:18:45 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:18:45.671235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad6e59-e813-4b55-94fb-f55cbdc84fc5 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:45.792346 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad6e59-e813-4b55-94fb-f55cbdc84fc5 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:45.830055 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8372af-f387-40e7-97ad-bf8a8709d242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:45.830694 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8372af-f387-40e7-97ad-bf8a8709d242 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:45.830975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8372af-f387-40e7-97ad-bf8a8709d242 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:45.831252 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8372af-f387-40e7-97ad-bf8a8709d242 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:45.842584 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8372af-f387-40e7-97ad-bf8a8709d242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:45.844263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8372af-f387-40e7-97ad-bf8a8709d242 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:45.858915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-8d8372af-f387-40e7-97ad-bf8a8709d242 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:45.858915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:45.858915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:45.858915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:45.858915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:45.858915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:45.858915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:45.858915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:45.858915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:45.858915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:45.858915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:45.858915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:45.858915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:45.858915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:45.858915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:45.858915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:45.859912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:45.859912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:45.859912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:45.859912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:45.859912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:45.859912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:45.859912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:45.859912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:45.859912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:45.859912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:45.859912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:45.859912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:45.859912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:45.859912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:45.859912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:45.859912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:45.859912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:45.860836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:45.860836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:45.860836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:45.860836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 453/908] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:45.865908 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46c352-baca-4d81-8e0e-b1399f0bfcc0 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:45.968002 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46c352-baca-4d81-8e0e-b1399f0bfcc0 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:46.004316 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bbde8-cda8-425f-b697-62317b5800af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:46.005325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bbde8-cda8-425f-b697-62317b5800af None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:46.005811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bbde8-cda8-425f-b697-62317b5800af None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:46.006347 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bbde8-cda8-425f-b697-62317b5800af None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:46.007195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bbde8-cda8-425f-b697-62317b5800af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:46.007706 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bbde8-cda8-425f-b697-62317b5800af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:46.009321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bbde8-cda8-425f-b697-62317b5800af None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:46.021289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 456/909] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:46.033209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8372af-f387-40e7-97ad-bf8a8709d242 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:46.082826 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:46.139966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8372af-f387-40e7-97ad-bf8a8709d242 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:46.175619 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7432a9-854e-4f90-b85c-7edddda363e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:46.176501 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7432a9-854e-4f90-b85c-7edddda363e0 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:46.176951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7432a9-854e-4f90-b85c-7edddda363e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:46.177403 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7432a9-854e-4f90-b85c-7edddda363e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:46.184910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7432a9-854e-4f90-b85c-7edddda363e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:46.186542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7432a9-854e-4f90-b85c-7edddda363e0 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:46.198718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-1b7432a9-854e-4f90-b85c-7edddda363e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:46.198718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:46.198718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:46.198718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:46.198718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:46.198718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:46.198718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:46.198718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:46.198718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:46.198718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:46.198718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:46.198718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:46.198718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:46.198718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:46.198718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:46.199845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:46.199845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:46.199845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:46.199845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:46.199845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:46.199845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:46.199845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:46.199845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:46.199845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:46.199845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:46.199845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:46.199845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:46.199845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:46.199845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:46.199845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:46.199845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:46.199845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:46.201110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:46.201110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:46.201110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:46.201110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:46.201110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:46.201110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:46.201110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:46.201110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:46.203288 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 454/910] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:46.209190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bbde8-cda8-425f-b697-62317b5800af None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:46.242454 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:46.317924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bbde8-cda8-425f-b697-62317b5800af None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:46.354545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9918d3-182f-475a-b3d0-0c5d690527bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:46.355390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9918d3-182f-475a-b3d0-0c5d690527bb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:46.355810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9918d3-182f-475a-b3d0-0c5d690527bb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:46.356355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9918d3-182f-475a-b3d0-0c5d690527bb None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:46.357252 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9918d3-182f-475a-b3d0-0c5d690527bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:46.357697 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9918d3-182f-475a-b3d0-0c5d690527bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:46.359267 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9918d3-182f-475a-b3d0-0c5d690527bb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:46.371551 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 457/911] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:46.377243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7432a9-854e-4f90-b85c-7edddda363e0 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:46.485648 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7432a9-854e-4f90-b85c-7edddda363e0 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:46.493662 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:46.524421 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d87be3-7ccd-42ed-8fbe-51a8b47f5d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:46.525029 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d87be3-7ccd-42ed-8fbe-51a8b47f5d64 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:46.525337 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d87be3-7ccd-42ed-8fbe-51a8b47f5d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:46.525609 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d87be3-7ccd-42ed-8fbe-51a8b47f5d64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:46.526451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d87be3-7ccd-42ed-8fbe-51a8b47f5d64 None admin] RBAC: Adding query filter params (role.id=ccde793bf8ca438a97cf89d53e0a0f1e, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=4ae237601b4a4aab9e969fedb5ee37d7) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:46.526758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d87be3-7ccd-42ed-8fbe-51a8b47f5d64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:46.528291 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d87be3-7ccd-42ed-8fbe-51a8b47f5d64 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:46.534455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 455/912] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:18:46 2022] GET /identity/v3/role_assignments?user.id=4ae237601b4a4aab9e969fedb5ee37d7&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=ccde793bf8ca438a97cf89d53e0a0f1e => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:46.770862 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d9932-67c2-42c1-ad3b-8cbf69c45e6c None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:46.771220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d9932-67c2-42c1-ad3b-8cbf69c45e6c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:46.771419 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d9932-67c2-42c1-ad3b-8cbf69c45e6c None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:46.773291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 458/913] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:46.779341 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cb87f-1c71-465b-b5b9-3135e22c342e None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:46.779626 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cb87f-1c71-465b-b5b9-3135e22c342e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:46.779898 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cb87f-1c71-465b-b5b9-3135e22c342e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:46.798323 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-837cb87f-1c71-465b-b5b9-3135e22c342e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:46.799514 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-837cb87f-1c71-465b-b5b9-3135e22c342e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:46.875757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 456/914] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:46.963042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a657c-b61c-4ac8-8276-32a8f0ddc96a None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:46.963699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a657c-b61c-4ac8-8276-32a8f0ddc96a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:46.964183 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a657c-b61c-4ac8-8276-32a8f0ddc96a None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:46.965987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 459/915] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:46.971413 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00520e6b-656f-454a-ad8a-eac282114665 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:46.971953 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00520e6b-656f-454a-ad8a-eac282114665 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:46.972457 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00520e6b-656f-454a-ad8a-eac282114665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:46.996475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-00520e6b-656f-454a-ad8a-eac282114665 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:46.997774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-00520e6b-656f-454a-ad8a-eac282114665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:47.067685 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 457/916] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:47.151074 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a657c-b61c-4ac8-8276-32a8f0ddc96a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:47.244292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:47.254509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a657c-b61c-4ac8-8276-32a8f0ddc96a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:47.289474 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b430f31-088e-42a7-9621-ae748d648eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:47.290666 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b430f31-088e-42a7-9621-ae748d648eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:47.290946 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b430f31-088e-42a7-9621-ae748d648eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:47.291222 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b430f31-088e-42a7-9621-ae748d648eb1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:47.291808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b430f31-088e-42a7-9621-ae748d648eb1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:47.293746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b430f31-088e-42a7-9621-ae748d648eb1 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:47.302513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-4b430f31-088e-42a7-9621-ae748d648eb1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 10:18:47.302513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:47.302513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:47.302513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:47.302513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:47.302513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:47.302513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:47.302513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:47.302513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 10:18:47.302513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:47.302513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:47.302513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:47.302513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:47.302513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:47.302513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:47.302513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:47.302513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:47.303563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:47.303563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:47.303563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:47.303563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:47.303563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:47.303563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:47.303563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:47.303563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:47.303563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:47.303563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:47.303563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:47.303563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:47.303563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:47.303563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:47.303563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:47.303563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:47.304438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:47.304438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:47.304438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:47.304438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:47.304438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:47.304438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:47.304438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:47.304438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:47.304438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:47.304438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:47.304438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:47.304438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:47.304438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 10:18:47.304438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:47.304438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 460/917] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:18:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:47.310102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00520e6b-656f-454a-ad8a-eac282114665 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:47.340769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:47.343565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b430f31-088e-42a7-9621-ae748d648eb1 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:47.419528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00520e6b-656f-454a-ad8a-eac282114665 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:47.459345 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa5428-8ce0-4682-9612-3f56120142d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:47.459986 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa5428-8ce0-4682-9612-3f56120142d6 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:47.460316 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa5428-8ce0-4682-9612-3f56120142d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:47.460693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa5428-8ce0-4682-9612-3f56120142d6 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:47.461514 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa5428-8ce0-4682-9612-3f56120142d6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:47.462471 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa5428-8ce0-4682-9612-3f56120142d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:47.463950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa5428-8ce0-4682-9612-3f56120142d6 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:47.468496 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b430f31-088e-42a7-9621-ae748d648eb1 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:47.473891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 458/918] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:18:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:47.475218 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa5428-8ce0-4682-9612-3f56120142d6 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:47.505439 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686056f-f787-40a9-8d8d-f13170dab766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:47.506069 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686056f-f787-40a9-8d8d-f13170dab766 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:47.506348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686056f-f787-40a9-8d8d-f13170dab766 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:47.506620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686056f-f787-40a9-8d8d-f13170dab766 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:47.507202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686056f-f787-40a9-8d8d-f13170dab766 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:47.509861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686056f-f787-40a9-8d8d-f13170dab766 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:47.519175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-0686056f-f787-40a9-8d8d-f13170dab766 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 10:18:47.519175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:47.519175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:47.519175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:47.519175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:47.519175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:47.519175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:47.519175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:47.519175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 10:18:47.519175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:47.519175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:47.519175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:47.519175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:47.519175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:47.519175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:47.519175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:47.519175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:47.520420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:47.520420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:47.520420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:47.520420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:47.520420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:47.520420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:47.520420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:47.520420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:47.520420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:47.520420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:47.520420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:47.520420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:47.520420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:47.520420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:47.520420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:47.520420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:47.521429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:47.521429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:47.521429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:47.521429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:47.521429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:47.521429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:47.521429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:47.521429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:47.521429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:47.521429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:47.521429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:47.521429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:47.521429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 10:18:47.521429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:47.525200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 461/919] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 19 10:18:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:47.527363 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686056f-f787-40a9-8d8d-f13170dab766 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:47.549658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:47.581201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:47.590430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa5428-8ce0-4682-9612-3f56120142d6 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:47.638475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ad410-c8bc-4d05-8d06-bcb8cf86d5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:47.639374 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ad410-c8bc-4d05-8d06-bcb8cf86d5d5 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:47.639867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ad410-c8bc-4d05-8d06-bcb8cf86d5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:47.640389 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ad410-c8bc-4d05-8d06-bcb8cf86d5d5 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:47.641248 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ad410-c8bc-4d05-8d06-bcb8cf86d5d5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:47.641772 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ad410-c8bc-4d05-8d06-bcb8cf86d5d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:47.643407 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ad410-c8bc-4d05-8d06-bcb8cf86d5d5 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:47.653880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686056f-f787-40a9-8d8d-f13170dab766 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:47.654247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 459/920] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Sep 19 10:18:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:47.655733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ad410-c8bc-4d05-8d06-bcb8cf86d5d5 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:47.691648 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f93ea-b33c-422e-8abd-1bec1f8e3472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:47.692519 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f93ea-b33c-422e-8abd-1bec1f8e3472 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:47.692803 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f93ea-b33c-422e-8abd-1bec1f8e3472 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:47.693100 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f93ea-b33c-422e-8abd-1bec1f8e3472 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:47.694618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f93ea-b33c-422e-8abd-1bec1f8e3472 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:47.694944 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f93ea-b33c-422e-8abd-1bec1f8e3472 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:47.696852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f93ea-b33c-422e-8abd-1bec1f8e3472 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:47.702593 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 462/921] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 19 10:18:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:47.707706 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f93ea-b33c-422e-8abd-1bec1f8e3472 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:47.770317 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ad410-c8bc-4d05-8d06-bcb8cf86d5d5 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:47.803548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:47.815010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149dbfe-0c56-44ee-8fcf-7e796baca39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:47.821652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149dbfe-0c56-44ee-8fcf-7e796baca39e None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:47.821652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149dbfe-0c56-44ee-8fcf-7e796baca39e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:47.821652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149dbfe-0c56-44ee-8fcf-7e796baca39e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:47.821652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149dbfe-0c56-44ee-8fcf-7e796baca39e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:47.821652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149dbfe-0c56-44ee-8fcf-7e796baca39e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:47.824582 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f93ea-b33c-422e-8abd-1bec1f8e3472 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:47.865691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18fa24-dbbf-42c2-8d43-4c6134c0f113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:47.866426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18fa24-dbbf-42c2-8d43-4c6134c0f113 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:47.866777 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18fa24-dbbf-42c2-8d43-4c6134c0f113 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:47.867125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18fa24-dbbf-42c2-8d43-4c6134c0f113 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:47.868394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18fa24-dbbf-42c2-8d43-4c6134c0f113 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:47.868762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18fa24-dbbf-42c2-8d43-4c6134c0f113 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:47.870242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18fa24-dbbf-42c2-8d43-4c6134c0f113 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:47.876044 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 463/922] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 19 10:18:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:47.881699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18fa24-dbbf-42c2-8d43-4c6134c0f113 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:47.886902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 460/923] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:18:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:18:47.968702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45280f38-84fb-47f5-8c72-e52d81f896df None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:47.969438 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45280f38-84fb-47f5-8c72-e52d81f896df None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:47.969934 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45280f38-84fb-47f5-8c72-e52d81f896df None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:47.971804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 461/924] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:18:47.978300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1729de-2cfd-4f1a-b8c8-77bba26521c4 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:47.978936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1729de-2cfd-4f1a-b8c8-77bba26521c4 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:47.979414 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1729de-2cfd-4f1a-b8c8-77bba26521c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:47.998534 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-2d1729de-2cfd-4f1a-b8c8-77bba26521c4 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:48.000111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-2d1729de-2cfd-4f1a-b8c8-77bba26521c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:48.005127 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18fa24-dbbf-42c2-8d43-4c6134c0f113 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:48.011450 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:48.053265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5959eef-b79a-4c09-b983-d496311af28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:48.053875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5959eef-b79a-4c09-b983-d496311af28c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:48.054155 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5959eef-b79a-4c09-b983-d496311af28c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:48.054425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5959eef-b79a-4c09-b983-d496311af28c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:48.055205 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5959eef-b79a-4c09-b983-d496311af28c None admin] RBAC: Adding query filter params (interface=public, service_id=d236936d3cd046d391bec828551c0b81, region_id=RegionOne) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:48.055457 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5959eef-b79a-4c09-b983-d496311af28c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:48.056880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5959eef-b79a-4c09-b983-d496311af28c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:48.062966 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 464/925] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Sep 19 10:18:47 2022] GET /identity/v3/endpoints?service_id=d236936d3cd046d391bec828551c0b81&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:48.068570 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5959eef-b79a-4c09-b983-d496311af28c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:48.078376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:48.088303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 462/926] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:18:48.186788 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5959eef-b79a-4c09-b983-d496311af28c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:48.223496 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8927231-8b3c-490e-a15b-6ddb469b520c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:48.224497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8927231-8b3c-490e-a15b-6ddb469b520c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:48.224959 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8927231-8b3c-490e-a15b-6ddb469b520c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:48.225448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8927231-8b3c-490e-a15b-6ddb469b520c None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:48.226239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8927231-8b3c-490e-a15b-6ddb469b520c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:48.228376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8927231-8b3c-490e-a15b-6ddb469b520c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:48.235151 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 465/927] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Sep 19 10:18:48 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:18:48.361997 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1729de-2cfd-4f1a-b8c8-77bba26521c4 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:48.473489 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1729de-2cfd-4f1a-b8c8-77bba26521c4 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:48.507705 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4bc593-ca9a-4f0f-88a8-7c7df14d895e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:48.508779 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bc593-ca9a-4f0f-88a8-7c7df14d895e None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:48.509080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bc593-ca9a-4f0f-88a8-7c7df14d895e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:48.509359 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bc593-ca9a-4f0f-88a8-7c7df14d895e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:48.509941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4bc593-ca9a-4f0f-88a8-7c7df14d895e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:48.511312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4bc593-ca9a-4f0f-88a8-7c7df14d895e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:48.520148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-7f4bc593-ca9a-4f0f-88a8-7c7df14d895e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 10:18:48.520148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:48.520148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:48.520148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:48.520148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:48.520148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:48.520148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:48.520148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:48.520148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 10:18:48.520148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:48.520148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:48.520148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:48.520148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:48.520148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:48.520148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:48.520148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:48.521149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:48.521149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:48.521149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:48.521149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:48.521149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:48.521149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:48.521149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:48.521149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:48.521149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:48.521149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:48.521149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:48.521149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:48.521149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:48.521149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:48.521149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:48.521149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:48.521149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:48.522080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:48.522080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:48.522080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:48.522080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:48.522080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:48.522080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:48.522080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:48.522080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:48.522080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:48.522080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:48.522080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:48.522080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:48.522080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 10:18:48.522080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:48.522080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 463/928] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Sep 19 10:18:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:48.531102 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8927231-8b3c-490e-a15b-6ddb469b520c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:48.606404 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:48.642887 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8927231-8b3c-490e-a15b-6ddb469b520c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:48.679872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff12178-195f-4b39-8970-65c4b5c79cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:48.680880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff12178-195f-4b39-8970-65c4b5c79cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:48.681391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff12178-195f-4b39-8970-65c4b5c79cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:48.682126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff12178-195f-4b39-8970-65c4b5c79cd2 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:48.682961 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff12178-195f-4b39-8970-65c4b5c79cd2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:48.683466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff12178-195f-4b39-8970-65c4b5c79cd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:48.685018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff12178-195f-4b39-8970-65c4b5c79cd2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:48.690455 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 466/929] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 19 10:18:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:48.696372 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4bc593-ca9a-4f0f-88a8-7c7df14d895e None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:48.708933 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:48.738779 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b866c8-6085-46df-b908-b3be29a996ec None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:48.739418 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b866c8-6085-46df-b908-b3be29a996ec None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:48.739914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b866c8-6085-46df-b908-b3be29a996ec None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:48.741779 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 467/930] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:18:48.749619 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a2aba-b8ec-4e70-bfd9-b8de53bfd330 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:48.750186 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a2aba-b8ec-4e70-bfd9-b8de53bfd330 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:48.750675 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a2aba-b8ec-4e70-bfd9-b8de53bfd330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:48.769377 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c32a2aba-b8ec-4e70-bfd9-b8de53bfd330 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:48.770838 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c32a2aba-b8ec-4e70-bfd9-b8de53bfd330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:48.822120 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4bc593-ca9a-4f0f-88a8-7c7df14d895e None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:48.862399 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 468/931] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:18:48.866724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a8a8e-5e7c-4883-b8e3-0d9534f33bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:48.867610 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a8a8e-5e7c-4883-b8e3-0d9534f33bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:48.868145 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a8a8e-5e7c-4883-b8e3-0d9534f33bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:48.868647 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a8a8e-5e7c-4883-b8e3-0d9534f33bd7 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:48.869493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a8a8e-5e7c-4883-b8e3-0d9534f33bd7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:48.870008 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a8a8e-5e7c-4883-b8e3-0d9534f33bd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:48.871656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a8a8e-5e7c-4883-b8e3-0d9534f33bd7 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:48.877339 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 464/932] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 19 10:18:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:48.886448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a2aba-b8ec-4e70-bfd9-b8de53bfd330 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:48.955797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:48.994445 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a2aba-b8ec-4e70-bfd9-b8de53bfd330 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:49.031702 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6b166-4439-452b-9a8b-277b8700f683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:49.032231 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6b166-4439-452b-9a8b-277b8700f683 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:49.032564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6b166-4439-452b-9a8b-277b8700f683 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:49.032842 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6b166-4439-452b-9a8b-277b8700f683 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:49.033601 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6b166-4439-452b-9a8b-277b8700f683 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:49.034962 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6b166-4439-452b-9a8b-277b8700f683 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:49.142467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 469/933] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:18:48 2022] POST /identity/v3/endpoints => generated 347 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:18:49.142983 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a8a8e-5e7c-4883-b8e3-0d9534f33bd7 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:49.229278 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:49.266886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a8a8e-5e7c-4883-b8e3-0d9534f33bd7 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:49.313494 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979e93c-5aea-4e5a-b3b0-913f9a5fd687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:49.314051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979e93c-5aea-4e5a-b3b0-913f9a5fd687 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:49.314329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979e93c-5aea-4e5a-b3b0-913f9a5fd687 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:49.314601 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979e93c-5aea-4e5a-b3b0-913f9a5fd687 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:49.327155 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979e93c-5aea-4e5a-b3b0-913f9a5fd687 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:49.332658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-6979e93c-5aea-4e5a-b3b0-913f9a5fd687 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 10:18:49.332658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:49.332658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:49.332658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:49.332658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:49.332658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:49.332658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:49.332658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:49.332658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:49.332658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:49.332658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:49.332658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:49.332658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:49.332658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:49.332658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:49.332658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:49.333671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:49.333671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:49.333671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:49.333671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:49.333671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:49.333671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:49.333671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:49.333671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:49.333671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:49.333671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:49.333671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:49.333671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:49.333671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:49.333671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:49.333671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:49.333671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:49.334537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:49.334537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:49.334537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:49.334537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:49.334537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:49.334537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:49.334537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:49.334537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:49.334537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:49.334537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:49.334537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 10:18:49.334537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:49.334537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 465/934] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Sep 19 10:18:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:18:49.341854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6b166-4439-452b-9a8b-277b8700f683 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:49.453375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6b166-4439-452b-9a8b-277b8700f683 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:49.474778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:49.489974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9483c-f276-4bc6-b6f1-dc1f57e786da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:49.490584 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9483c-f276-4bc6-b6f1-dc1f57e786da None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:49.490861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9483c-f276-4bc6-b6f1-dc1f57e786da None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:49.491139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9483c-f276-4bc6-b6f1-dc1f57e786da None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:49.491802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9483c-f276-4bc6-b6f1-dc1f57e786da None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:49.492108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9483c-f276-4bc6-b6f1-dc1f57e786da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:49.493581 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9483c-f276-4bc6-b6f1-dc1f57e786da None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:49.503267 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 470/935] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Sep 19 10:18:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:49.510192 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979e93c-5aea-4e5a-b3b0-913f9a5fd687 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:49.622001 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979e93c-5aea-4e5a-b3b0-913f9a5fd687 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:49.650043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:49.658586 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedabbd9-475f-4b8d-be3e-a60face07afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:49.659216 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedabbd9-475f-4b8d-be3e-a60face07afa None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:49.659494 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedabbd9-475f-4b8d-be3e-a60face07afa None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:49.659769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedabbd9-475f-4b8d-be3e-a60face07afa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:49.671100 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedabbd9-475f-4b8d-be3e-a60face07afa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:49.674524 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedabbd9-475f-4b8d-be3e-a60face07afa None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:49.687297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-fedabbd9-475f-4b8d-be3e-a60face07afa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:49.687297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:49.687297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:49.687297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:49.687297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:49.687297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:49.687297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:49.687297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:49.687297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:49.687297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:49.687297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:49.687297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:49.687297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:49.687297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:49.687297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:49.687297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:49.688256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:49.688256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:49.688256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:49.688256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:49.688256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:49.688256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:49.688256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:49.688256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:49.688256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:49.688256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:49.688256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:49.688256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:49.688256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:49.688256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:49.688256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:49.688256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:49.688256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:49.689243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:49.689243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:49.689243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:49.689243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 466/936] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:49.697220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9483c-f276-4bc6-b6f1-dc1f57e786da None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:49.800838 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9483c-f276-4bc6-b6f1-dc1f57e786da None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:49.835416 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e7c19-45c5-4a4b-b440-484fde08c6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:49.836029 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e7c19-45c5-4a4b-b440-484fde08c6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:49.836882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e7c19-45c5-4a4b-b440-484fde08c6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:49.837197 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e7c19-45c5-4a4b-b440-484fde08c6e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:49.837845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e7c19-45c5-4a4b-b440-484fde08c6e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:49.838149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e7c19-45c5-4a4b-b440-484fde08c6e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:49.839501 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e7c19-45c5-4a4b-b440-484fde08c6e6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:49.852030 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 471/937] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:49.862188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedabbd9-475f-4b8d-be3e-a60face07afa None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:49.966973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedabbd9-475f-4b8d-be3e-a60face07afa None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:50.001257 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be7d2e-68a4-480d-a550-7e7f63f0433c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:50.001886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be7d2e-68a4-480d-a550-7e7f63f0433c None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:50.002164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be7d2e-68a4-480d-a550-7e7f63f0433c None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:50.002434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be7d2e-68a4-480d-a550-7e7f63f0433c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:50.010583 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be7d2e-68a4-480d-a550-7e7f63f0433c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:50.012039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be7d2e-68a4-480d-a550-7e7f63f0433c None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:50.023711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-21be7d2e-68a4-480d-a550-7e7f63f0433c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 10:18:50.023711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:50.023711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:50.023711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:50.023711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:50.023711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:50.023711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:50.023711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:50.023711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:50.023711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:50.023711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:50.023711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:50.023711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:50.023711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:50.023711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:50.023711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:50.025293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:50.025293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:50.025293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:50.025293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:50.025293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:50.025293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:50.025293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:50.025293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:50.025293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:50.025293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:50.025293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:50.025293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:50.025293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:50.025293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:50.025293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:50.025293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:50.025293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:50.025293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:50.026247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:50.026247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:50.026247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:50.026247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:50.026247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:50.026247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:50.026247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:50.026247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:50.026247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 10:18:50.026247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:50.026247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 467/938] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 19 10:18:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:50.033214 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e7c19-45c5-4a4b-b440-484fde08c6e6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:50.083055 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:50.148463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e7c19-45c5-4a4b-b440-484fde08c6e6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:50.183677 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c2808f-f071-4ed4-9141-462ef25a5285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:50.184284 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2808f-f071-4ed4-9141-462ef25a5285 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:50.184563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2808f-f071-4ed4-9141-462ef25a5285 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:50.184844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2808f-f071-4ed4-9141-462ef25a5285 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:50.185593 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c2808f-f071-4ed4-9141-462ef25a5285 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:50.185903 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c2808f-f071-4ed4-9141-462ef25a5285 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:50.187313 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c2808f-f071-4ed4-9141-462ef25a5285 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:50.205893 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 472/939] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Sep 19 10:18:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:50.211751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be7d2e-68a4-480d-a550-7e7f63f0433c None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:50.238012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab7db9-e5c9-4f88-9396-3ab3b58b31b0 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:50.238569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab7db9-e5c9-4f88-9396-3ab3b58b31b0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:50.239011 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab7db9-e5c9-4f88-9396-3ab3b58b31b0 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:50.240867 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 473/940] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:18:50.246757 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865e3e8-af6e-498e-99ac-f2070318cabc None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:50.247202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865e3e8-af6e-498e-99ac-f2070318cabc None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:50.247820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865e3e8-af6e-498e-99ac-f2070318cabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:50.256833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:50.269901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c865e3e8-af6e-498e-99ac-f2070318cabc None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:50.271102 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c865e3e8-af6e-498e-99ac-f2070318cabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:50.322310 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be7d2e-68a4-480d-a550-7e7f63f0433c None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:50.357236 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590f039-0da0-4cf4-a822-4c8a341b3596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:50.358067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590f039-0da0-4cf4-a822-4c8a341b3596 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:50.358443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590f039-0da0-4cf4-a822-4c8a341b3596 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:50.358722 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 474/941] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:18:50.358838 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590f039-0da0-4cf4-a822-4c8a341b3596 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:50.374201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590f039-0da0-4cf4-a822-4c8a341b3596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:50.375718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590f039-0da0-4cf4-a822-4c8a341b3596 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:50.390667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-c590f039-0da0-4cf4-a822-4c8a341b3596 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:50.390667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:50.390667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:50.390667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:50.390667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:50.390667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:50.390667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:50.390667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:50.390667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:50.390667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:50.390667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:50.390667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:50.390667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:50.390667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:50.390667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:50.390667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:50.391620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:50.391620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:50.391620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:50.391620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:50.391620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:50.391620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:50.391620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:50.391620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:50.391620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:50.391620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:50.391620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:50.391620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:50.391620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:50.391620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:50.391620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:50.391620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:50.391620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:50.392543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:50.392543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:50.392543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:50.392543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 468/942] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:50.399146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865e3e8-af6e-498e-99ac-f2070318cabc None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:50.435999 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:50.513556 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865e3e8-af6e-498e-99ac-f2070318cabc None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:50.549583 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d7e64-f356-4860-b05f-3722cb880c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:50.550192 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d7e64-f356-4860-b05f-3722cb880c0b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:50.550476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d7e64-f356-4860-b05f-3722cb880c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:50.550752 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d7e64-f356-4860-b05f-3722cb880c0b None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:50.551403 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d7e64-f356-4860-b05f-3722cb880c0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:50.551709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d7e64-f356-4860-b05f-3722cb880c0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:50.553085 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d7e64-f356-4860-b05f-3722cb880c0b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:50.565394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 475/943] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:50.570908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590f039-0da0-4cf4-a822-4c8a341b3596 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:50.606296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a604ccb-0033-4fb1-99a3-0996867ecfa9 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:50.606620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a604ccb-0033-4fb1-99a3-0996867ecfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:50.606858 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a604ccb-0033-4fb1-99a3-0996867ecfa9 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:50.608429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 476/944] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:18:50.613511 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d861cb-4763-4584-a963-4a305af3f9dd None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:50.613797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d861cb-4763-4584-a963-4a305af3f9dd None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:50.614077 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d861cb-4763-4584-a963-4a305af3f9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:50.634204 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-69d861cb-4763-4584-a963-4a305af3f9dd None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:50.635411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-69d861cb-4763-4584-a963-4a305af3f9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:50.668250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:50.677574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590f039-0da0-4cf4-a822-4c8a341b3596 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:50.713259 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 477/945] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:50.713259 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d861cb-4763-4584-a963-4a305af3f9dd None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:50.716574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfff3f-ae87-4ade-a0ce-cebc570e9081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:50.717459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfff3f-ae87-4ade-a0ce-cebc570e9081 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:50.717947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfff3f-ae87-4ade-a0ce-cebc570e9081 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:50.718401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfff3f-ae87-4ade-a0ce-cebc570e9081 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:50.726256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcfff3f-ae87-4ade-a0ce-cebc570e9081 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:50.728001 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcfff3f-ae87-4ade-a0ce-cebc570e9081 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:50.731955 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:50.740785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-9fcfff3f-ae87-4ade-a0ce-cebc570e9081 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:50.740785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:50.740785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:50.740785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:50.740785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:50.740785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:50.740785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:50.740785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:50.740785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:50.740785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:50.740785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:50.740785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:50.740785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:50.740785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:50.740785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:50.745905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:50.745905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:50.745905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:50.745905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:50.745905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:50.745905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:50.745905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:50.745905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:50.745905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:50.745905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:50.745905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:50.745905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:50.745905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:50.745905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:50.745905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:50.745905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:50.745905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:50.746821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:50.746821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:50.746821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:50.746821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:50.746821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:50.746821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:50.746821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:50.746821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:50.746821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 469/946] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:50.748593 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfff3f-ae87-4ade-a0ce-cebc570e9081 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:50.831886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d861cb-4763-4584-a963-4a305af3f9dd None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:50.877591 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f40e4e-c1d7-4f09-8aff-8eba2872e55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:50.878163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfff3f-ae87-4ade-a0ce-cebc570e9081 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:50.878280 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f40e4e-c1d7-4f09-8aff-8eba2872e55d None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:50.878667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f40e4e-c1d7-4f09-8aff-8eba2872e55d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:50.879005 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f40e4e-c1d7-4f09-8aff-8eba2872e55d None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:50.879828 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f40e4e-c1d7-4f09-8aff-8eba2872e55d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:50.881444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f40e4e-c1d7-4f09-8aff-8eba2872e55d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:50.884652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: WARNING keystone.server.flask.common [None req-49f40e4e-c1d7-4f09-8aff-8eba2872e55d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 19 10:18:50.914037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c5d40-0410-4f0f-be55-5c81463b566a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:50.914684 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c5d40-0410-4f0f-be55-5c81463b566a None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:50.914960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c5d40-0410-4f0f-be55-5c81463b566a None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:50.915232 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c5d40-0410-4f0f-be55-5c81463b566a None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:50.916055 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c5d40-0410-4f0f-be55-5c81463b566a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:50.916364 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c5d40-0410-4f0f-be55-5c81463b566a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:50.917819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c5d40-0410-4f0f-be55-5c81463b566a None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:50.931215 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 470/947] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:50.937111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c5d40-0410-4f0f-be55-5c81463b566a None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:50.940796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.sql.core [None req-49f40e4e-c1d7-4f09-8aff-8eba2872e55d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 19 10:18:50.940796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 19 10:18:50.940796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [parameters: {'id': '975c3d5471d744c2b110260decb88192', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 19 10:18:50.940796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81562) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 19 10:18:50.957417 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:50.961000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-49f40e4e-c1d7-4f09-8aff-8eba2872e55d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 975c3d5471d744c2b110260decb88192.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 975c3d5471d744c2b110260decb88192. Sep 19 10:18:50.961000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:50.961000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 10:18:50.961000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 10:18:50.961000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 10:18:50.961000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 10:18:50.961000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 10:18:50.961000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 10:18:50.961000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 10:18:50.961000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application conn.query(q) Sep 19 10:18:50.961000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 10:18:50.961000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 10:18:50.961000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 10:18:50.961883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application result.read() Sep 19 10:18:50.961883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 10:18:50.961883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 10:18:50.961883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 10:18:50.961883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 10:18:50.961883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 10:18:50.961883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 10:18:50.961883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 10:18:50.961883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 10:18:50.961883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 19 10:18:50.961883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:50.961883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 19 10:18:50.961883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:50.961883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:50.961883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 19 10:18:50.961883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 19 10:18:50.961883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 19 10:18:50.961883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 19 10:18:50.962859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 10:18:50.962859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application next(self.gen) Sep 19 10:18:50.962859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 19 10:18:50.962859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application yield resource Sep 19 10:18:50.962859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 10:18:50.962859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application next(self.gen) Sep 19 10:18:50.962859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 19 10:18:50.962859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self.session.rollback() Sep 19 10:18:50.962859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 10:18:50.962859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self.force_reraise() Sep 19 10:18:50.962859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 10:18:50.962859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise self.value Sep 19 10:18:50.962859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 19 10:18:50.962859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 19 10:18:50.962859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 19 10:18:50.962859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application session.commit() Sep 19 10:18:50.962859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 19 10:18:50.963879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 19 10:18:50.963879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 19 10:18:50.963879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._prepare_impl() Sep 19 10:18:50.963879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 19 10:18:50.963879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self.session.flush() Sep 19 10:18:50.963879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 19 10:18:50.963879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._flush(objects) Sep 19 10:18:50.963879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 19 10:18:50.963879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 19 10:18:50.963879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 19 10:18:50.963879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application compat.raise_( Sep 19 10:18:50.963879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 10:18:50.963879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception Sep 19 10:18:50.963879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 19 10:18:50.963879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application flush_context.execute() Sep 19 10:18:50.963879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 19 10:18:50.963879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rec.execute(self) Sep 19 10:18:50.964806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 19 10:18:50.964806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 19 10:18:50.964806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 19 10:18:50.964806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 19 10:18:50.964806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 19 10:18:50.964806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 19 10:18:50.964806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 19 10:18:50.964806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 19 10:18:50.964806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 19 10:18:50.964806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 19 10:18:50.964806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 19 10:18:50.964806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 19 10:18:50.964806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 19 10:18:50.964806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 19 10:18:50.964806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 19 10:18:50.967248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 19 10:18:50.967248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 10:18:50.967248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception Sep 19 10:18:50.967248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 10:18:50.967248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 10:18:50.967248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 10:18:50.967248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 10:18:50.967248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 10:18:50.967248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 10:18:50.967248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 10:18:50.967248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application conn.query(q) Sep 19 10:18:50.967248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 10:18:50.967248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 10:18:50.967248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 10:18:50.967248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application result.read() Sep 19 10:18:50.967248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 10:18:50.967248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 10:18:50.969620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 10:18:50.969620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 10:18:50.969620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 10:18:50.969620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 10:18:50.969620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 10:18:50.969620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 10:18:50.969620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 19 10:18:50.969620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 19 10:18:50.969620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [parameters: {'id': '975c3d5471d744c2b110260decb88192', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 19 10:18:50.969620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 19 10:18:50.969620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:50.969620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:50.969620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:50.969620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:50.969620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 19 10:18:50.971604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 19 10:18:50.971604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 19 10:18:50.971604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 19 10:18:50.971604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 19 10:18:50.971604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:50.971604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:50.971604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:50.971604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:50.971604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:50.971604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:50.971604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:50.971604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:50.971604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:50.971604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:50.971604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:50.971604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:50.971604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:50.975048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:50.975048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 19 10:18:50.975048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 19 10:18:50.975048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:50.975048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:50.975048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 19 10:18:50.975048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 19 10:18:50.975048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 975c3d5471d744c2b110260decb88192. Sep 19 10:18:50.975048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:50.975048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 478/948] 10.222.0.42 () {60 vars in 1140 bytes} [Mon Sep 19 10:18:50 2022] POST /identity/v3/projects => generated 262 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:50.975048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f40e4e-c1d7-4f09-8aff-8eba2872e55d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:51.062005 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:51.064265 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c5d40-0410-4f0f-be55-5c81463b566a None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:51.086938 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f40e4e-c1d7-4f09-8aff-8eba2872e55d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:51.099344 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9f923-78a2-4ff9-93f4-022ed264d665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:51.100082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9f923-78a2-4ff9-93f4-022ed264d665 None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:51.100301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9f923-78a2-4ff9-93f4-022ed264d665 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:51.100541 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9f923-78a2-4ff9-93f4-022ed264d665 None admin] PATH_INFO: `/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/4ae237601b4a4aab9e969fedb5ee37d7/roles/ccde793bf8ca438a97cf89d53e0a0f1e` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:51.102789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9f923-78a2-4ff9-93f4-022ed264d665 None admin] RBAC: Authorizing `identity:create_grant(project_id=e45b8fbe79994e7585095328a7ec9260, user_id=4ae237601b4a4aab9e969fedb5ee37d7, role_id=ccde793bf8ca438a97cf89d53e0a0f1e)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:51.104217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9f923-78a2-4ff9-93f4-022ed264d665 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:51.122806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bd596-20e8-43c7-b6ec-187b8f5a4a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:51.123450 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bd596-20e8-43c7-b6ec-187b8f5a4a6d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:51.123729 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bd596-20e8-43c7-b6ec-187b8f5a4a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:51.124000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bd596-20e8-43c7-b6ec-187b8f5a4a6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:51.130823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bd596-20e8-43c7-b6ec-187b8f5a4a6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:51.132457 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bd596-20e8-43c7-b6ec-187b8f5a4a6d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:51.144154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-b28bd596-20e8-43c7-b6ec-187b8f5a4a6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:51.144154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:51.144154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:51.144154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:51.144154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:51.144154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:51.144154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:51.144154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:51.144154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:51.144154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:51.144154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:51.144154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:51.144154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:51.144154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:51.144154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:51.145242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:51.145242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:51.145242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:51.145242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:51.145242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:51.145242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:51.145242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:51.145242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:51.145242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:51.145242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:51.145242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:51.145242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:51.145242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:51.145242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:51.145242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:51.145242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:51.145242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:51.146417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:51.146417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:51.146417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:51.146417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:51.146417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:51.146417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:51.146417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:51.146417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:51.149204 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 479/949] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:51.150249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bd596-20e8-43c7-b6ec-187b8f5a4a6d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:51.173089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 471/950] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:18:50 2022] PUT /identity/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/4ae237601b4a4aab9e969fedb5ee37d7/roles/ccde793bf8ca438a97cf89d53e0a0f1e => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:18:51.173089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9f923-78a2-4ff9-93f4-022ed264d665 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:51.253669 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:51.280272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bd596-20e8-43c7-b6ec-187b8f5a4a6d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:51.291544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9f923-78a2-4ff9-93f4-022ed264d665 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:51.335541 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af9ac9-6856-432e-9f38-8f298a52dd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:51.336336 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af9ac9-6856-432e-9f38-8f298a52dd1a None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:51.336678 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af9ac9-6856-432e-9f38-8f298a52dd1a None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:51.337098 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af9ac9-6856-432e-9f38-8f298a52dd1a None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:51.337452 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac59a79-a5a9-438b-b66f-27c183ee8301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:51.337874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af9ac9-6856-432e-9f38-8f298a52dd1a None admin] RBAC: Adding query filter params (name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:51.338216 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac59a79-a5a9-438b-b66f-27c183ee8301 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:51.338342 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af9ac9-6856-432e-9f38-8f298a52dd1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:51.338630 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac59a79-a5a9-438b-b66f-27c183ee8301 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:51.338974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac59a79-a5a9-438b-b66f-27c183ee8301 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:51.339634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac59a79-a5a9-438b-b66f-27c183ee8301 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:51.340011 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af9ac9-6856-432e-9f38-8f298a52dd1a None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:51.341166 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac59a79-a5a9-438b-b66f-27c183ee8301 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:51.345373 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:51.354543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-dac59a79-a5a9-438b-b66f-27c183ee8301 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 10:18:51.354543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:51.354543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:51.354543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:51.354543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:51.354543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:51.354543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:51.354543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:51.354543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 10:18:51.354543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:51.354543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:51.354543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:51.354543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:51.354543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:51.354543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:51.354543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:51.354543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:51.355982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:51.355982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:51.355982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:51.355982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:51.355982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:51.355982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:51.355982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:51.355982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:51.355982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:51.355982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:51.355982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:51.355982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:51.355982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:51.355982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:51.355982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:51.355982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:51.357139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:51.357139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:51.357139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:51.357139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:51.357139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:51.357139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:51.357139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:51.357139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:51.357139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:51.357139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:51.357139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:51.357139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:51.357139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 10:18:51.357139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:51.363930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 480/951] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 19 10:18:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:51.369115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 472/952] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 19 10:18:51 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:51.370383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac59a79-a5a9-438b-b66f-27c183ee8301 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:51.429740 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820a4d2-f968-4881-9ca4-acf0aa42bf5a None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:51.431056 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820a4d2-f968-4881-9ca4-acf0aa42bf5a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:51.431480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820a4d2-f968-4881-9ca4-acf0aa42bf5a None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:51.434359 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 473/953] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:18:51.440439 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58fc7b-9810-4a10-8b42-f2aa0dbf33ac None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:51.440804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58fc7b-9810-4a10-8b42-f2aa0dbf33ac None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:51.441378 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58fc7b-9810-4a10-8b42-f2aa0dbf33ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:51.467547 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-4c58fc7b-9810-4a10-8b42-f2aa0dbf33ac None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:51.469805 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4c58fc7b-9810-4a10-8b42-f2aa0dbf33ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:51.492232 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac59a79-a5a9-438b-b66f-27c183ee8301 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:51.531433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955e12a-7751-42f8-9ff1-07babaee73db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:51.532112 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955e12a-7751-42f8-9ff1-07babaee73db None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:51.532363 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955e12a-7751-42f8-9ff1-07babaee73db None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:51.532700 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955e12a-7751-42f8-9ff1-07babaee73db None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:51.533396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955e12a-7751-42f8-9ff1-07babaee73db None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:51.533751 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955e12a-7751-42f8-9ff1-07babaee73db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:51.535182 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955e12a-7751-42f8-9ff1-07babaee73db None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:51.545750 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 481/954] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 19 10:18:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:51.551858 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955e12a-7751-42f8-9ff1-07babaee73db None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:51.573159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 474/955] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:18:51.626251 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:51.666247 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955e12a-7751-42f8-9ff1-07babaee73db None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:51.701944 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb93e9-4c2d-42bb-b35c-6576e29c4e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:51.702639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb93e9-4c2d-42bb-b35c-6576e29c4e6a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:51.702983 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb93e9-4c2d-42bb-b35c-6576e29c4e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:51.703320 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb93e9-4c2d-42bb-b35c-6576e29c4e6a None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:51.704019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb93e9-4c2d-42bb-b35c-6576e29c4e6a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:51.704392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb93e9-4c2d-42bb-b35c-6576e29c4e6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:51.705922 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb93e9-4c2d-42bb-b35c-6576e29c4e6a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:51.711761 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 482/956] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 19 10:18:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:51.718606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58fc7b-9810-4a10-8b42-f2aa0dbf33ac None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:51.788702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:51.829104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58fc7b-9810-4a10-8b42-f2aa0dbf33ac None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:51.856153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb93e9-4c2d-42bb-b35c-6576e29c4e6a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:51.867202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441262d-bc7b-4c1b-85b9-f01f0e31f2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:51.868364 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441262d-bc7b-4c1b-85b9-f01f0e31f2e0 None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:51.868647 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441262d-bc7b-4c1b-85b9-f01f0e31f2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:51.868979 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441262d-bc7b-4c1b-85b9-f01f0e31f2e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:51.869701 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441262d-bc7b-4c1b-85b9-f01f0e31f2e0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:51.871558 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441262d-bc7b-4c1b-85b9-f01f0e31f2e0 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:51.960698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 475/957] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:18:51 2022] POST /identity/v3/endpoints => generated 349 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:18:51.975294 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb93e9-4c2d-42bb-b35c-6576e29c4e6a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:52.007769 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:52.029293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25f3df-ef21-4096-a3d7-a8cf58d23598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:52.030190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25f3df-ef21-4096-a3d7-a8cf58d23598 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:52.030620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25f3df-ef21-4096-a3d7-a8cf58d23598 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:52.031069 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25f3df-ef21-4096-a3d7-a8cf58d23598 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:52.031808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25f3df-ef21-4096-a3d7-a8cf58d23598 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:52.033471 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25f3df-ef21-4096-a3d7-a8cf58d23598 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:52.043885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-0b25f3df-ef21-4096-a3d7-a8cf58d23598 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 10:18:52.043885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:52.043885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:52.043885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:52.043885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:52.043885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:52.043885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:52.043885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:52.043885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 10:18:52.043885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:52.043885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:52.043885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:52.043885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:52.043885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:52.043885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:52.043885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:52.043885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:52.045202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:52.045202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:52.045202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:52.045202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:52.045202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:52.045202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:52.045202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:52.045202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:52.045202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:52.045202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:52.045202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:52.045202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:52.045202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:52.045202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:52.045202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:52.045202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:52.046270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:52.046270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:52.046270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:52.046270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:52.046270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:52.046270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:52.046270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:52.046270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:52.046270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:52.046270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:52.046270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:52.046270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:52.046270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 10:18:52.046270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:52.048770 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 483/958] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:52.055232 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441262d-bc7b-4c1b-85b9-f01f0e31f2e0 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:52.174524 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441262d-bc7b-4c1b-85b9-f01f0e31f2e0 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:52.195136 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:52.214327 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511d246-2455-4c0e-a346-9e51a42affc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:52.214866 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511d246-2455-4c0e-a346-9e51a42affc3 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:52.215144 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511d246-2455-4c0e-a346-9e51a42affc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:52.215418 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511d246-2455-4c0e-a346-9e51a42affc3 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:52.216050 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511d246-2455-4c0e-a346-9e51a42affc3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:52.216351 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511d246-2455-4c0e-a346-9e51a42affc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:52.217802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511d246-2455-4c0e-a346-9e51a42affc3 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:52.223957 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 476/959] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 19 10:18:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:52.229489 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25f3df-ef21-4096-a3d7-a8cf58d23598 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:52.334651 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25f3df-ef21-4096-a3d7-a8cf58d23598 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:52.374203 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ec9ab-da60-44ff-8c4e-5a36ea01d290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:52.374812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ec9ab-da60-44ff-8c4e-5a36ea01d290 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:52.375092 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ec9ab-da60-44ff-8c4e-5a36ea01d290 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:52.375367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ec9ab-da60-44ff-8c4e-5a36ea01d290 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:52.375995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ec9ab-da60-44ff-8c4e-5a36ea01d290 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:52.376300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ec9ab-da60-44ff-8c4e-5a36ea01d290 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:52.377699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ec9ab-da60-44ff-8c4e-5a36ea01d290 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:52.382540 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 484/960] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 19 10:18:52 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:53.495914 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff0f6f-0ac7-4c3e-9e7b-d1c2c00cc909 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:53.496480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff0f6f-0ac7-4c3e-9e7b-d1c2c00cc909 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:53.496480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff0f6f-0ac7-4c3e-9e7b-d1c2c00cc909 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:53.498079 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 477/961] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:53.503631 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85757fca-7f2d-4553-8f78-d8db4ad18f96 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:53.504030 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85757fca-7f2d-4553-8f78-d8db4ad18f96 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:53.504223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85757fca-7f2d-4553-8f78-d8db4ad18f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:53.523316 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-85757fca-7f2d-4553-8f78-d8db4ad18f96 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:53.525402 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-85757fca-7f2d-4553-8f78-d8db4ad18f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:53.572145 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:53.592861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 485/962] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:18:53.767628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd9bb7-79ff-46bd-8b08-1cc1ae3009c5 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:53.767826 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd9bb7-79ff-46bd-8b08-1cc1ae3009c5 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:53.768095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd9bb7-79ff-46bd-8b08-1cc1ae3009c5 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:53.769700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 478/963] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:53.777120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e32e1-b2de-4040-8f3c-90c8dae453c8 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:53.777120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e32e1-b2de-4040-8f3c-90c8dae453c8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:53.777120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e32e1-b2de-4040-8f3c-90c8dae453c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:53.795064 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5c6e32e1-b2de-4040-8f3c-90c8dae453c8 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:53.796543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5c6e32e1-b2de-4040-8f3c-90c8dae453c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:53.865274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 486/964] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:53.890548 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd9bb7-79ff-46bd-8b08-1cc1ae3009c5 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:53.999870 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd9bb7-79ff-46bd-8b08-1cc1ae3009c5 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:54.035638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b42b73-8341-4f82-8210-4494b1ede8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:54.036193 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b42b73-8341-4f82-8210-4494b1ede8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:54.036472 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b42b73-8341-4f82-8210-4494b1ede8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:54.036743 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b42b73-8341-4f82-8210-4494b1ede8b8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:54.048511 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b42b73-8341-4f82-8210-4494b1ede8b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:18:54.053399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-43b42b73-8341-4f82-8210-4494b1ede8b8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 10:18:54.053399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:54.053399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:54.053399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:54.053399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:54.053399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:54.053399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:54.053399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:54.053399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:54.053399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:54.053399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:54.053399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:54.053399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:18:54.053399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:18:54.053399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:18:54.053399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:18:54.054354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:18:54.054354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:18:54.054354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:18:54.054354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:18:54.054354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:54.054354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:54.054354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:54.054354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:54.054354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:54.054354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:54.054354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:54.054354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:54.054354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:54.054354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:54.054354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:54.054354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:54.055261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:54.055261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:54.055261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:54.055261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:54.055261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:18:54.055261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:18:54.055261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:18:54.055261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:18:54.055261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:18:54.055261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:18:54.055261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 10:18:54.055261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:54.055261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 479/965] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Sep 19 10:18:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:54.061969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e32e1-b2de-4040-8f3c-90c8dae453c8 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:54.170140 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b42b73-8341-4f82-8210-4494b1ede8b8 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:54.177427 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e32e1-b2de-4040-8f3c-90c8dae453c8 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:54.192621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:54.227338 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:54.229868 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5a6e7-c223-42a3-94a1-96db5e7ef8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:54.230916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5a6e7-c223-42a3-94a1-96db5e7ef8a5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:54.230916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5a6e7-c223-42a3-94a1-96db5e7ef8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:54.231091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5a6e7-c223-42a3-94a1-96db5e7ef8a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:54.232078 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5a6e7-c223-42a3-94a1-96db5e7ef8a5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:54.232279 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5a6e7-c223-42a3-94a1-96db5e7ef8a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:54.234213 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5a6e7-c223-42a3-94a1-96db5e7ef8a5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:54.245215 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 487/966] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Sep 19 10:18:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:54.247397 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850347f-11c6-4df2-8f13-ea9d6be363f7 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:54.248227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850347f-11c6-4df2-8f13-ea9d6be363f7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:54.248711 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850347f-11c6-4df2-8f13-ea9d6be363f7 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:54.250331 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 488/967] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:54.251099 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850347f-11c6-4df2-8f13-ea9d6be363f7 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:54.294426 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b42b73-8341-4f82-8210-4494b1ede8b8 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:54.329476 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae1f4a-dc91-4bb2-bf56-3d9d566e67ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:54.330086 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae1f4a-dc91-4bb2-bf56-3d9d566e67ff None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:54.330370 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae1f4a-dc91-4bb2-bf56-3d9d566e67ff None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:54.330672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae1f4a-dc91-4bb2-bf56-3d9d566e67ff None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:54.331315 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae1f4a-dc91-4bb2-bf56-3d9d566e67ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:54.332760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae1f4a-dc91-4bb2-bf56-3d9d566e67ff None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:54.335377 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: WARNING keystone.server.flask.common [None req-a4ae1f4a-dc91-4bb2-bf56-3d9d566e67ff None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 19 10:18:54.358332 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850347f-11c6-4df2-8f13-ea9d6be363f7 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:54.368782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.sql.core [None req-a4ae1f4a-dc91-4bb2-bf56-3d9d566e67ff None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 19 10:18:54.368782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 19 10:18:54.368782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [parameters: {'user_id': '9fb5eeb7a22a4ce5964ee31ad668a3d8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 19 10:18:54.368782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81563) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 19 10:18:54.377010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-a4ae1f4a-dc91-4bb2-bf56-3d9d566e67ff None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 19 10:18:54.377010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:54.377010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 10:18:54.377010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 10:18:54.377010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 10:18:54.377010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 10:18:54.377010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 10:18:54.377010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 10:18:54.377010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 10:18:54.377010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application conn.query(q) Sep 19 10:18:54.377010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 10:18:54.377010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 10:18:54.377010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 10:18:54.377010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application result.read() Sep 19 10:18:54.377010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 10:18:54.377960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 10:18:54.377960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 10:18:54.377960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 10:18:54.377960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 10:18:54.377960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 10:18:54.377960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 10:18:54.377960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 10:18:54.377960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 19 10:18:54.377960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:54.377960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 19 10:18:54.377960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:54.377960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:54.377960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 19 10:18:54.377960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 19 10:18:54.377960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 19 10:18:54.377960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 19 10:18:54.377960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 10:18:54.377960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application next(self.gen) Sep 19 10:18:54.382201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 19 10:18:54.382201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application yield resource Sep 19 10:18:54.382201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 10:18:54.382201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application next(self.gen) Sep 19 10:18:54.382201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 19 10:18:54.382201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self.session.rollback() Sep 19 10:18:54.382201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 10:18:54.382201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self.force_reraise() Sep 19 10:18:54.382201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 10:18:54.382201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise self.value Sep 19 10:18:54.382201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 19 10:18:54.382201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 19 10:18:54.382201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 19 10:18:54.382201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application session.commit() Sep 19 10:18:54.382201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 19 10:18:54.382201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 19 10:18:54.382201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 19 10:18:54.383696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._prepare_impl() Sep 19 10:18:54.383696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 19 10:18:54.383696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self.session.flush() Sep 19 10:18:54.383696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 19 10:18:54.383696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._flush(objects) Sep 19 10:18:54.383696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 19 10:18:54.383696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 19 10:18:54.383696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 19 10:18:54.383696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application compat.raise_( Sep 19 10:18:54.383696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 10:18:54.383696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception Sep 19 10:18:54.383696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 19 10:18:54.383696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application flush_context.execute() Sep 19 10:18:54.383696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 19 10:18:54.383696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rec.execute(self) Sep 19 10:18:54.383696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 19 10:18:54.383696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 19 10:18:54.384596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 19 10:18:54.384596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 19 10:18:54.384596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 19 10:18:54.384596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 19 10:18:54.384596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 19 10:18:54.384596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 19 10:18:54.384596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 19 10:18:54.384596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 19 10:18:54.384596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 19 10:18:54.384596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 19 10:18:54.384596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 19 10:18:54.384596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 19 10:18:54.384596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 19 10:18:54.384596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 19 10:18:54.384596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 10:18:54.384596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception Sep 19 10:18:54.385713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 10:18:54.385713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 10:18:54.385713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 10:18:54.385713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 10:18:54.385713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 10:18:54.385713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 10:18:54.385713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 10:18:54.385713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application conn.query(q) Sep 19 10:18:54.385713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 10:18:54.385713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 10:18:54.385713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 10:18:54.385713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application result.read() Sep 19 10:18:54.385713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 10:18:54.385713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 10:18:54.385713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 10:18:54.385713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 10:18:54.386751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 10:18:54.386751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 10:18:54.386751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 10:18:54.386751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 10:18:54.386751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 19 10:18:54.386751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 19 10:18:54.386751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [parameters: {'user_id': '9fb5eeb7a22a4ce5964ee31ad668a3d8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 19 10:18:54.386751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 19 10:18:54.386751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:54.386751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:54.386751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:54.386751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:54.386751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:54.386751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:54.386751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:54.387845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:54.387845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:54.387845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:54.387845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:54.387845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:54.387845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:54.387845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:54.387845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 19 10:18:54.387845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 19 10:18:54.387845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:54.387845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:54.387845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:54.387845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:54.387845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:54.387845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:54.387845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 19 10:18:54.387845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 19 10:18:54.389329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 19 10:18:54.389329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 19 10:18:54.389329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 19 10:18:54.389329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 19 10:18:54.389329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 19 10:18:54.389329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:54.389329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 480/968] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Sep 19 10:18:54 2022] POST /identity/v3/users => generated 160 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:18:54.389329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05594925-c1cc-4f13-91fa-c20f3db36722 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:54.389329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05594925-c1cc-4f13-91fa-c20f3db36722 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:54.389329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05594925-c1cc-4f13-91fa-c20f3db36722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:54.401606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-05594925-c1cc-4f13-91fa-c20f3db36722 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:54.404658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-05594925-c1cc-4f13-91fa-c20f3db36722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:54.405136 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f610a6-db0e-4d02-a057-e3564907dc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:54.405766 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f610a6-db0e-4d02-a057-e3564907dc35 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:54.406043 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f610a6-db0e-4d02-a057-e3564907dc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:54.406319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f610a6-db0e-4d02-a057-e3564907dc35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:54.417538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f610a6-db0e-4d02-a057-e3564907dc35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:54.418985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f610a6-db0e-4d02-a057-e3564907dc35 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:54.435067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-d0f610a6-db0e-4d02-a057-e3564907dc35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:54.435067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:54.435067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:54.435067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:54.435067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:54.435067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:54.435067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:54.435067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:54.435067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:54.435067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:54.435067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:54.435067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:54.435067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:54.435067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:54.435067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:54.435067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:54.436079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:54.436079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:54.436079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:54.436079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:54.436079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:54.436079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:54.436079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:54.436079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:54.436079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:54.436079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:54.436079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:54.436079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:54.436079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:54.436079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:54.436079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:54.436079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:54.436079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:54.441891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:54.441891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:54.441891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:54.441891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 489/969] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:54.441891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f610a6-db0e-4d02-a057-e3564907dc35 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:54.470302 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:54.491528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 481/970] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:54.492786 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05594925-c1cc-4f13-91fa-c20f3db36722 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:54.534021 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:54.566422 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f610a6-db0e-4d02-a057-e3564907dc35 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:54.605820 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05594925-c1cc-4f13-91fa-c20f3db36722 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:54.615227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d620a28-8fe7-40de-86bd-d84cb4545a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:54.615858 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d620a28-8fe7-40de-86bd-d84cb4545a28 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:54.616503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d620a28-8fe7-40de-86bd-d84cb4545a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:54.616781 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d620a28-8fe7-40de-86bd-d84cb4545a28 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:54.625635 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d620a28-8fe7-40de-86bd-d84cb4545a28 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:54.627066 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d620a28-8fe7-40de-86bd-d84cb4545a28 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:54.639319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-4d620a28-8fe7-40de-86bd-d84cb4545a28 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 10:18:54.639319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:54.639319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:54.639319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:54.639319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:54.639319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:54.639319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:54.639319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:54.639319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:54.639319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:54.639319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:54.639319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:54.639319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:54.639319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:54.639319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:54.639319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:54.640298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:54.640298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:54.640298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:54.640298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:54.640298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:54.640298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:54.640298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:54.640298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:54.640298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:54.640298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:54.640298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:54.640298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:54.640298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:54.640298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:54.640298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:54.640298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:54.640298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:54.640298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:54.641339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:54.641339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:54.641339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:54.641339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:54.641339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:54.641339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:54.641339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:54.641339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:54.641339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 10:18:54.641339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:54.641339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 490/971] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:18:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:54.642500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5324b-ce04-4c43-b5d5-e5ead69f8518 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:54.642711 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5324b-ce04-4c43-b5d5-e5ead69f8518 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:54.643534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5324b-ce04-4c43-b5d5-e5ead69f8518 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:54.649078 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 491/972] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:54.649078 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5324b-ce04-4c43-b5d5-e5ead69f8518 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:54.649744 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c768dc9-0134-4799-ad45-65fe0c51301e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:54.650350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c768dc9-0134-4799-ad45-65fe0c51301e None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:54.650631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c768dc9-0134-4799-ad45-65fe0c51301e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:54.650967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c768dc9-0134-4799-ad45-65fe0c51301e None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:54.651620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c768dc9-0134-4799-ad45-65fe0c51301e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:54.651927 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c768dc9-0134-4799-ad45-65fe0c51301e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:54.653532 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c768dc9-0134-4799-ad45-65fe0c51301e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:54.672668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 482/973] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:54.672668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c04a3a-93a0-4cfe-bda1-bde9ea6b84a9 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:54.672668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c04a3a-93a0-4cfe-bda1-bde9ea6b84a9 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:54.672668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c04a3a-93a0-4cfe-bda1-bde9ea6b84a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:54.689323 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-c9c04a3a-93a0-4cfe-bda1-bde9ea6b84a9 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:54.690510 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c9c04a3a-93a0-4cfe-bda1-bde9ea6b84a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:54.738868 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:54.755154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5324b-ce04-4c43-b5d5-e5ead69f8518 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:54.765530 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 483/974] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:54.766393 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c04a3a-93a0-4cfe-bda1-bde9ea6b84a9 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:54.795534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c29a1f-1719-4d9e-adf3-ad85089b67fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:54.796542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c29a1f-1719-4d9e-adf3-ad85089b67fb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:54.796827 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c29a1f-1719-4d9e-adf3-ad85089b67fb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:54.797123 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c29a1f-1719-4d9e-adf3-ad85089b67fb None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:54.797972 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c29a1f-1719-4d9e-adf3-ad85089b67fb None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:54.798278 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c29a1f-1719-4d9e-adf3-ad85089b67fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:54.799613 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:54.799725 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c29a1f-1719-4d9e-adf3-ad85089b67fb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:54.842677 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 492/975] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Sep 19 10:18:54 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:54.843518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c29a1f-1719-4d9e-adf3-ad85089b67fb None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:54.894064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c04a3a-93a0-4cfe-bda1-bde9ea6b84a9 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:54.934997 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e82cd9-3ccc-47ae-a1ed-871951ba77d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:54.935960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e82cd9-3ccc-47ae-a1ed-871951ba77d5 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:54.936421 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e82cd9-3ccc-47ae-a1ed-871951ba77d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:54.937024 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e82cd9-3ccc-47ae-a1ed-871951ba77d5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:54.946074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e82cd9-3ccc-47ae-a1ed-871951ba77d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:54.947766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e82cd9-3ccc-47ae-a1ed-871951ba77d5 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:54.963328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-95e82cd9-3ccc-47ae-a1ed-871951ba77d5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 10:18:54.963328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:54.963328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:54.963328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:54.963328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:54.963328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:54.963328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:54.963328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:54.963328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:54.963328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:54.963328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:54.963328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:54.963328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:18:54.963328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:18:54.963328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:18:54.963328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:18:54.964632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:54.964632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:54.964632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:18:54.964632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:54.964632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:18:54.964632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:18:54.964632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:54.964632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:54.964632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:54.964632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:54.964632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:54.964632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:54.964632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:54.964632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:54.964632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:54.964632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:54.964632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:54.964632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:54.965834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:54.965834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:54.965834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:18:54.965834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:18:54.965834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:18:54.965834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:18:54.965834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:18:54.965834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:18:54.965834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 10:18:54.965834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:54.967210 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 484/976] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 19 10:18:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:54.972605 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e82cd9-3ccc-47ae-a1ed-871951ba77d5 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:54.980082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c29a1f-1719-4d9e-adf3-ad85089b67fb None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:55.020682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd7262-4512-4d01-b5aa-21979855abb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:55.021372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd7262-4512-4d01-b5aa-21979855abb5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:55.021655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd7262-4512-4d01-b5aa-21979855abb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:55.022185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd7262-4512-4d01-b5aa-21979855abb5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:55.023434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd7262-4512-4d01-b5aa-21979855abb5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:55.025279 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd7262-4512-4d01-b5aa-21979855abb5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:55.036950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-62cd7262-4512-4d01-b5aa-21979855abb5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 10:18:55.036950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:55.036950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:55.036950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:55.036950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:55.036950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:55.036950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:55.036950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:55.036950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 10:18:55.036950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:55.036950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:55.036950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:55.036950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:55.036950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:55.036950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:55.036950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:55.036950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:55.038335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:55.038335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:55.038335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:55.038335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:55.038335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:55.038335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:55.038335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:55.038335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:55.038335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:55.038335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:55.038335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:55.038335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:55.038335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:55.038335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:55.038335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:55.038335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:55.039821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:55.039821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:55.039821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:55.039821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:55.039821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:55.039821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:55.039821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:55.039821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:55.039821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:55.039821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:55.039821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:55.039821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:55.039821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 10:18:55.039821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:55.039821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 493/977] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 19 10:18:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:55.040796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd7262-4512-4d01-b5aa-21979855abb5 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:55.053225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:55.078534 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:55.096905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e82cd9-3ccc-47ae-a1ed-871951ba77d5 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:55.135667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa84da5-5ba8-4f42-a6dc-3f565806b9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:55.136329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa84da5-5ba8-4f42-a6dc-3f565806b9e7 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:55.136613 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa84da5-5ba8-4f42-a6dc-3f565806b9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:55.136884 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa84da5-5ba8-4f42-a6dc-3f565806b9e7 None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:55.137652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa84da5-5ba8-4f42-a6dc-3f565806b9e7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:55.137956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa84da5-5ba8-4f42-a6dc-3f565806b9e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:55.139389 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa84da5-5ba8-4f42-a6dc-3f565806b9e7 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:55.163709 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 485/978] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Sep 19 10:18:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:55.164647 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa84da5-5ba8-4f42-a6dc-3f565806b9e7 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:55.167895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd7262-4512-4d01-b5aa-21979855abb5 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:55.201680 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb6d84-0386-4516-9a70-88aa5f2db6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:55.202284 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb6d84-0386-4516-9a70-88aa5f2db6bd None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:55.202568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb6d84-0386-4516-9a70-88aa5f2db6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:55.202841 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb6d84-0386-4516-9a70-88aa5f2db6bd None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:55.203492 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb6d84-0386-4516-9a70-88aa5f2db6bd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:55.204852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb6d84-0386-4516-9a70-88aa5f2db6bd None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:55.260421 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa84da5-5ba8-4f42-a6dc-3f565806b9e7 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:55.283457 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 494/979] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 19 10:18:55 2022] POST /identity/v3/services => generated 242 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:18:55.284363 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb6d84-0386-4516-9a70-88aa5f2db6bd None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:55.302478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac8797-2407-4f47-9bea-f2422f9ad046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:55.303113 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac8797-2407-4f47-9bea-f2422f9ad046 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:55.303392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac8797-2407-4f47-9bea-f2422f9ad046 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:55.303667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac8797-2407-4f47-9bea-f2422f9ad046 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:55.304311 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac8797-2407-4f47-9bea-f2422f9ad046 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:55.305271 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac8797-2407-4f47-9bea-f2422f9ad046 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:55.306699 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac8797-2407-4f47-9bea-f2422f9ad046 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:55.313876 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 486/980] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 19 10:18:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:55.319704 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac8797-2407-4f47-9bea-f2422f9ad046 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:55.323988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:55.370618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:55.412797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb6d84-0386-4516-9a70-88aa5f2db6bd None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:55.440845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac8797-2407-4f47-9bea-f2422f9ad046 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:55.458094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adee80-377d-4c55-839b-2c86c5bc7541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:55.458708 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adee80-377d-4c55-839b-2c86c5bc7541 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:55.458990 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adee80-377d-4c55-839b-2c86c5bc7541 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:55.459261 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adee80-377d-4c55-839b-2c86c5bc7541 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:55.482765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4efe10-97e2-44f6-b1e6-5b2744eea632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:55.483381 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4efe10-97e2-44f6-b1e6-5b2744eea632 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:55.483665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4efe10-97e2-44f6-b1e6-5b2744eea632 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:55.483940 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4efe10-97e2-44f6-b1e6-5b2744eea632 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:55.484599 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4efe10-97e2-44f6-b1e6-5b2744eea632 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:55.484904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4efe10-97e2-44f6-b1e6-5b2744eea632 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:55.486855 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4efe10-97e2-44f6-b1e6-5b2744eea632 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:55.492757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 487/981] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 19 10:18:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:55.499164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4efe10-97e2-44f6-b1e6-5b2744eea632 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:55.534970 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0adee80-377d-4c55-839b-2c86c5bc7541 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:55.536424 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0adee80-377d-4c55-839b-2c86c5bc7541 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:55.569998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:55.604729 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:55.607294 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4efe10-97e2-44f6-b1e6-5b2744eea632 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:55.641161 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 495/982] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:18:55 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:18:55.646322 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adee80-377d-4c55-839b-2c86c5bc7541 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:55.647711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6e448-5eab-4276-b675-b745f2ce2fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:55.648403 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6e448-5eab-4276-b675-b745f2ce2fce None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:55.648808 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6e448-5eab-4276-b675-b745f2ce2fce None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:55.649101 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6e448-5eab-4276-b675-b745f2ce2fce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:55.650679 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6e448-5eab-4276-b675-b745f2ce2fce None admin] RBAC: Adding query filter params (interface=admin, service_id=d236936d3cd046d391bec828551c0b81, region_id=RegionOne) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:55.650991 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6e448-5eab-4276-b675-b745f2ce2fce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:55.652364 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6e448-5eab-4276-b675-b745f2ce2fce None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:55.663154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 488/983] 10.222.0.42 () {56 vars in 1249 bytes} [Mon Sep 19 10:18:55 2022] GET /identity/v3/endpoints?service_id=d236936d3cd046d391bec828551c0b81&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:55.664190 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6e448-5eab-4276-b675-b745f2ce2fce None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:55.757260 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adee80-377d-4c55-839b-2c86c5bc7541 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:55.764510 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6e448-5eab-4276-b675-b745f2ce2fce None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:55.793313 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeaee72-7551-4bec-bcc4-9b672f04f2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:55.793978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeaee72-7551-4bec-bcc4-9b672f04f2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:55.794215 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeaee72-7551-4bec-bcc4-9b672f04f2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:55.794492 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeaee72-7551-4bec-bcc4-9b672f04f2f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:55.797444 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:55.800983 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e2796-5f1e-4f1b-9dff-04fd59a3f657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:55.801642 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e2796-5f1e-4f1b-9dff-04fd59a3f657 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:55.801924 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e2796-5f1e-4f1b-9dff-04fd59a3f657 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:55.802195 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e2796-5f1e-4f1b-9dff-04fd59a3f657 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:55.802800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e2796-5f1e-4f1b-9dff-04fd59a3f657 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:55.804155 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e2796-5f1e-4f1b-9dff-04fd59a3f657 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:55.805551 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeaee72-7551-4bec-bcc4-9b672f04f2f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:55.806979 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeaee72-7551-4bec-bcc4-9b672f04f2f2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:55.812821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 489/984] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Sep 19 10:18:55 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:18:55.828082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-5aeaee72-7551-4bec-bcc4-9b672f04f2f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:55.828082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:55.828082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:55.828082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:55.828082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:55.828082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:55.828082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:55.828082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:55.828082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:55.828082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:55.828082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:55.828082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:55.828082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:18:55.828082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:18:55.828082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:18:55.828082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:18:55.829188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:55.829188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:55.829188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:55.829188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:55.829188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:55.829188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:55.829188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:55.829188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:55.829188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:55.829188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:55.829188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:55.829188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:55.829188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:55.829188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:55.829188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:55.829188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:55.829188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:18:55.830135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:18:55.830135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:18:55.830135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:55.830135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 496/985] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:18:55.834987 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e2796-5f1e-4f1b-9dff-04fd59a3f657 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:55.945601 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e2796-5f1e-4f1b-9dff-04fd59a3f657 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:55.982205 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f652cfa-7588-4012-8e78-6d2430608b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:55.982882 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f652cfa-7588-4012-8e78-6d2430608b63 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:55.983104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f652cfa-7588-4012-8e78-6d2430608b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:55.983380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f652cfa-7588-4012-8e78-6d2430608b63 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:55.984028 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f652cfa-7588-4012-8e78-6d2430608b63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:55.984336 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f652cfa-7588-4012-8e78-6d2430608b63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:55.985744 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f652cfa-7588-4012-8e78-6d2430608b63 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:55.998608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 490/986] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:56.003845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeaee72-7551-4bec-bcc4-9b672f04f2f2 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:56.031737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:56.110032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeaee72-7551-4bec-bcc4-9b672f04f2f2 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:56.144603 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f5d15-ae59-48da-bccd-7c989fa88f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:56.145261 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f5d15-ae59-48da-bccd-7c989fa88f5a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:56.145539 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f5d15-ae59-48da-bccd-7c989fa88f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:56.145811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f5d15-ae59-48da-bccd-7c989fa88f5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:56.152799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f5d15-ae59-48da-bccd-7c989fa88f5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:56.154264 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f5d15-ae59-48da-bccd-7c989fa88f5a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:56.165375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-327f5d15-ae59-48da-bccd-7c989fa88f5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:56.165375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:56.165375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:56.165375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:56.165375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:56.165375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:56.165375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:56.165375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:56.165375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:56.165375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:56.165375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:56.165375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:56.165375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:18:56.165375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:18:56.165375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:18:56.166255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:18:56.166255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:56.166255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:56.166255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:56.166255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:56.166255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:56.166255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:56.166255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:56.166255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:56.166255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:56.166255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:56.166255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:56.166255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:56.166255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:56.166255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:56.166255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:56.166255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:56.167177 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:18:56.167177 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:18:56.167177 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:18:56.167177 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:18:56.167177 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:18:56.167177 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:18:56.167177 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:18:56.167177 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:56.167177 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 497/987] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:56.172140 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f652cfa-7588-4012-8e78-6d2430608b63 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:56.232021 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:56.276733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f652cfa-7588-4012-8e78-6d2430608b63 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:56.310611 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a03351-3944-4fd9-b31e-aea663e8430c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:56.311214 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a03351-3944-4fd9-b31e-aea663e8430c None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:56.311488 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a03351-3944-4fd9-b31e-aea663e8430c None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:56.311759 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a03351-3944-4fd9-b31e-aea663e8430c None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:56.312439 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a03351-3944-4fd9-b31e-aea663e8430c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:56.312743 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a03351-3944-4fd9-b31e-aea663e8430c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:56.314203 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a03351-3944-4fd9-b31e-aea663e8430c None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:56.326531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 491/988] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:18:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:56.332291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f5d15-ae59-48da-bccd-7c989fa88f5a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:56.413259 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:56.432868 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f5d15-ae59-48da-bccd-7c989fa88f5a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:56.467435 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce88f8-579c-49b3-a549-a5c474b21769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:56.468354 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce88f8-579c-49b3-a549-a5c474b21769 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:56.468840 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce88f8-579c-49b3-a549-a5c474b21769 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:56.469354 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce88f8-579c-49b3-a549-a5c474b21769 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:56.470276 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce88f8-579c-49b3-a549-a5c474b21769 None admin] RBAC: Adding query filter params (role.id=ccde793bf8ca438a97cf89d53e0a0f1e, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=4ae237601b4a4aab9e969fedb5ee37d7) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:56.470789 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce88f8-579c-49b3-a549-a5c474b21769 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:56.472428 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce88f8-579c-49b3-a549-a5c474b21769 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:56.484131 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 498/989] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:18:56 2022] GET /identity/v3/role_assignments?user.id=4ae237601b4a4aab9e969fedb5ee37d7&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=ccde793bf8ca438a97cf89d53e0a0f1e => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:57.489021 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fc942-df0d-4fca-8224-9daa7db33d1f None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:57.489558 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fc942-df0d-4fca-8224-9daa7db33d1f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:57.489676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fc942-df0d-4fca-8224-9daa7db33d1f None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:57.490976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 492/990] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:57.496544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307d7b9-9043-4a7c-83d2-f83194e15e80 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:57.496922 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307d7b9-9043-4a7c-83d2-f83194e15e80 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:57.497140 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307d7b9-9043-4a7c-83d2-f83194e15e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:57.515867 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6307d7b9-9043-4a7c-83d2-f83194e15e80 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:57.517084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6307d7b9-9043-4a7c-83d2-f83194e15e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:57.585256 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 499/991] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:57.857011 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fc942-df0d-4fca-8224-9daa7db33d1f None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:57.965547 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fc942-df0d-4fca-8224-9daa7db33d1f None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:57.979925 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:58.000093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c3b19-4437-4067-8d92-b8ec445a6f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:58.000993 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c3b19-4437-4067-8d92-b8ec445a6f51 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:58.007149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c3b19-4437-4067-8d92-b8ec445a6f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:58.007149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c3b19-4437-4067-8d92-b8ec445a6f51 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:58.007149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c3b19-4437-4067-8d92-b8ec445a6f51 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:58.007149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c3b19-4437-4067-8d92-b8ec445a6f51 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:58.012941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-a85c3b19-4437-4067-8d92-b8ec445a6f51 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 10:18:58.012941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:58.012941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:58.012941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:58.012941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:58.012941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:58.012941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:58.012941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:58.012941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 10:18:58.012941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:58.012941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:58.012941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:58.012941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:58.012941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:58.012941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:58.012941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:58.012941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:58.014147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:58.014147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:58.014147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:58.014147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:58.014147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:58.014147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:58.014147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:58.014147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:58.014147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:58.014147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:58.014147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:58.014147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:58.014147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:58.014147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:58.014147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:58.014147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:58.015188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:58.015188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:58.015188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:58.015188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:58.015188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:58.015188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:58.015188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:58.015188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:58.015188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:58.015188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:58.015188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:58.015188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:58.015188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 10:18:58.015188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:18:58.015972 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5faed4-2a6c-4396-ad09-08ca8bf6e97b None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:58.015972 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5faed4-2a6c-4396-ad09-08ca8bf6e97b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:58.015972 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5faed4-2a6c-4396-ad09-08ca8bf6e97b None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:58.015972 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 500/992] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:18:58.017485 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 493/993] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:18:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:58.020139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9207c9e-e824-437f-bf5a-e9f7e60cea9b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:58.020492 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9207c9e-e824-437f-bf5a-e9f7e60cea9b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:58.020839 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9207c9e-e824-437f-bf5a-e9f7e60cea9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:58.025584 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c3b19-4437-4067-8d92-b8ec445a6f51 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:58.041557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e9207c9e-e824-437f-bf5a-e9f7e60cea9b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:58.042602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e9207c9e-e824-437f-bf5a-e9f7e60cea9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:58.112546 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 501/994] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:18:58.142337 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c3b19-4437-4067-8d92-b8ec445a6f51 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:58.176778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02421636-abdc-4a24-9768-67ea8fd036fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:58.177408 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02421636-abdc-4a24-9768-67ea8fd036fa None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:58.177687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02421636-abdc-4a24-9768-67ea8fd036fa None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:58.177959 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02421636-abdc-4a24-9768-67ea8fd036fa None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:58.178578 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02421636-abdc-4a24-9768-67ea8fd036fa None admin] RBAC: Adding query filter params (name=compute) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:58.178879 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02421636-abdc-4a24-9768-67ea8fd036fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:58.180849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02421636-abdc-4a24-9768-67ea8fd036fa None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:58.185855 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 494/995] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 19 10:18:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:58.191324 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9207c9e-e824-437f-bf5a-e9f7e60cea9b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:58.194775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:58.289964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9207c9e-e824-437f-bf5a-e9f7e60cea9b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:58.323672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a083083-bcd4-48ca-b60e-71808c528b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:58.324281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a083083-bcd4-48ca-b60e-71808c528b3e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:58.324560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a083083-bcd4-48ca-b60e-71808c528b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:58.324832 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a083083-bcd4-48ca-b60e-71808c528b3e None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:58.325487 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a083083-bcd4-48ca-b60e-71808c528b3e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:58.325789 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a083083-bcd4-48ca-b60e-71808c528b3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:58.327137 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a083083-bcd4-48ca-b60e-71808c528b3e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:58.336628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 502/996] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 19 10:18:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:58.337781 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02421636-abdc-4a24-9768-67ea8fd036fa None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:58.384065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:58.386933 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a083083-bcd4-48ca-b60e-71808c528b3e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:58.444651 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02421636-abdc-4a24-9768-67ea8fd036fa None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:58.462550 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:58.483452 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0e582-c902-4bf8-ac0d-a1bff7306ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:58.484315 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0e582-c902-4bf8-ac0d-a1bff7306ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:58.484762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0e582-c902-4bf8-ac0d-a1bff7306ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:58.485224 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0e582-c902-4bf8-ac0d-a1bff7306ca2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:58.485920 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0e582-c902-4bf8-ac0d-a1bff7306ca2 None admin] RBAC: Adding query filter params (interface=public, service_id=bd019724bd1b4d2d9611592b58effcbb, region_id=RegionOne) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:58.486228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0e582-c902-4bf8-ac0d-a1bff7306ca2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:58.487625 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0e582-c902-4bf8-ac0d-a1bff7306ca2 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:58.497705 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a083083-bcd4-48ca-b60e-71808c528b3e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:58.498500 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 495/997] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Sep 19 10:18:58 2022] GET /identity/v3/endpoints?service_id=bd019724bd1b4d2d9611592b58effcbb&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:58.499298 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0e582-c902-4bf8-ac0d-a1bff7306ca2 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:58.534421 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de82a5-20c0-433a-90c9-7a43a06ae33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:58.535047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de82a5-20c0-433a-90c9-7a43a06ae33a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:58.535325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de82a5-20c0-433a-90c9-7a43a06ae33a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:58.535596 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de82a5-20c0-433a-90c9-7a43a06ae33a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:58.536176 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de82a5-20c0-433a-90c9-7a43a06ae33a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:58.537574 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de82a5-20c0-433a-90c9-7a43a06ae33a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:58.546411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-d3de82a5-20c0-433a-90c9-7a43a06ae33a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 10:18:58.546411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:58.546411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:58.546411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:58.546411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:58.546411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:58.546411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:58.546411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:58.546411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 10:18:58.546411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:58.546411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:58.546411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:58.546411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:58.546411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:58.546411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:58.546411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:58.546411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:58.547443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:58.547443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:58.547443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:58.547443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:58.547443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:58.547443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:58.547443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:58.547443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:58.547443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:58.547443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:58.547443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:58.547443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:58.547443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:58.547443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:58.547443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:58.547443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:58.548299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:58.548299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:58.548299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:58.548299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:58.548299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:58.548299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:58.548299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:58.548299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:58.548299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:58.548299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:58.548299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:58.548299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:58.548299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 10:18:58.548299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:58.548299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 503/998] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Sep 19 10:18:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:18:58.553260 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de82a5-20c0-433a-90c9-7a43a06ae33a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:58.614124 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0e582-c902-4bf8-ac0d-a1bff7306ca2 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:58.622696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:58.650064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea3ed1-97a0-4e21-bf5c-064644cc3514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:58.650596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea3ed1-97a0-4e21-bf5c-064644cc3514 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:58.650876 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea3ed1-97a0-4e21-bf5c-064644cc3514 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:58.651149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea3ed1-97a0-4e21-bf5c-064644cc3514 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:58.651748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea3ed1-97a0-4e21-bf5c-064644cc3514 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:58.653142 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea3ed1-97a0-4e21-bf5c-064644cc3514 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:58.659994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 496/999] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Sep 19 10:18:58 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:18:58.675191 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de82a5-20c0-433a-90c9-7a43a06ae33a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:58.696125 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4ae35-d6f9-445a-b238-457a9e5a917f None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:58.696400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4ae35-d6f9-445a-b238-457a9e5a917f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:58.696867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4ae35-d6f9-445a-b238-457a9e5a917f None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:58.698250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 497/1000] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:18:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:18:58.704009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419aa50-4b53-4945-8c8a-3de01ff66c78 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:58.704291 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419aa50-4b53-4945-8c8a-3de01ff66c78 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:58.704572 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419aa50-4b53-4945-8c8a-3de01ff66c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:58.711184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:58.716212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5ff1c-4f36-4855-a328-0776fbf2891f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:58.716838 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5ff1c-4f36-4855-a328-0776fbf2891f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:58.717235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5ff1c-4f36-4855-a328-0776fbf2891f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:58.717508 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5ff1c-4f36-4855-a328-0776fbf2891f None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:58.718154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5ff1c-4f36-4855-a328-0776fbf2891f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:58.718460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5ff1c-4f36-4855-a328-0776fbf2891f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:58.719867 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5ff1c-4f36-4855-a328-0776fbf2891f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:58.723896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-a419aa50-4b53-4945-8c8a-3de01ff66c78 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:18:58.725177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-a419aa50-4b53-4945-8c8a-3de01ff66c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:18:58.729248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 504/1001] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 19 10:18:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:58.731720 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5ff1c-4f36-4855-a328-0776fbf2891f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:58.812473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 498/1002] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:18:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:18:58.860620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5ff1c-4f36-4855-a328-0776fbf2891f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:58.895210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb68eb4-951f-4b30-a4ed-1049317f36a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:58.895820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb68eb4-951f-4b30-a4ed-1049317f36a5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:58.896100 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb68eb4-951f-4b30-a4ed-1049317f36a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:58.896370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb68eb4-951f-4b30-a4ed-1049317f36a5 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:58.897283 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb68eb4-951f-4b30-a4ed-1049317f36a5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:58.897587 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb68eb4-951f-4b30-a4ed-1049317f36a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:58.898971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb68eb4-951f-4b30-a4ed-1049317f36a5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:58.908629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 505/1003] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Sep 19 10:18:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:18:58.909939 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419aa50-4b53-4945-8c8a-3de01ff66c78 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:59.010595 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:59.013722 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419aa50-4b53-4945-8c8a-3de01ff66c78 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:59.047634 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4254774-1356-4e13-85ad-b233315e3433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:59.048243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4254774-1356-4e13-85ad-b233315e3433 None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:59.048519 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4254774-1356-4e13-85ad-b233315e3433 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:59.048788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4254774-1356-4e13-85ad-b233315e3433 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:59.049497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4254774-1356-4e13-85ad-b233315e3433 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:59.050859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4254774-1356-4e13-85ad-b233315e3433 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:59.080471 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb68eb4-951f-4b30-a4ed-1049317f36a5 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:59.118415 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 499/1004] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:18:58 2022] POST /identity/v3/endpoints => generated 325 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:18:59.146947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:59.196437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb68eb4-951f-4b30-a4ed-1049317f36a5 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:59.240966 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cd68e-eb5d-45d9-86d5-3243ccd65497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:59.241986 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cd68e-eb5d-45d9-86d5-3243ccd65497 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:59.242432 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cd68e-eb5d-45d9-86d5-3243ccd65497 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:59.242853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cd68e-eb5d-45d9-86d5-3243ccd65497 None admin] PATH_INFO: `/v3/services/image` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:59.243785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815cd68e-eb5d-45d9-86d5-3243ccd65497 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:59.245709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815cd68e-eb5d-45d9-86d5-3243ccd65497 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:59.255560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-815cd68e-eb5d-45d9-86d5-3243ccd65497 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 10:18:59.255560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:59.255560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:18:59.255560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:18:59.255560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:18:59.255560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:18:59.255560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:18:59.255560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:59.255560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 10:18:59.255560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:59.255560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:18:59.255560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:59.255560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:18:59.255560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:18:59.255560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:18:59.255560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:18:59.255560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:18:59.256850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:18:59.256850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:18:59.256850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:18:59.256850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:18:59.256850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:18:59.256850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:18:59.256850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:18:59.256850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:18:59.256850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:18:59.256850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:18:59.256850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:18:59.256850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:18:59.256850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:18:59.256850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:18:59.256850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:18:59.256850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:18:59.257826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:18:59.257826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:18:59.257826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:18:59.257826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:18:59.257826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:18:59.257826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:18:59.257826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:18:59.257826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:18:59.257826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:18:59.257826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:18:59.257826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:18:59.257826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:18:59.257826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 10:18:59.257826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:18:59.258979 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 506/1005] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Sep 19 10:18:59 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:18:59.264893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4254774-1356-4e13-85ad-b233315e3433 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:59.372614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4254774-1356-4e13-85ad-b233315e3433 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:59.407018 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c9fb9f-4e81-4943-9fea-ccb6be5ce922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:59.407629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c9fb9f-4e81-4943-9fea-ccb6be5ce922 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:59.407901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c9fb9f-4e81-4943-9fea-ccb6be5ce922 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:59.412588 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c9fb9f-4e81-4943-9fea-ccb6be5ce922 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:59.412588 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c9fb9f-4e81-4943-9fea-ccb6be5ce922 None admin] RBAC: Adding query filter params (name=image) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:59.412588 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c9fb9f-4e81-4943-9fea-ccb6be5ce922 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:59.412588 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c9fb9f-4e81-4943-9fea-ccb6be5ce922 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:59.415691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 500/1006] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:18:59.421179 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cd68e-eb5d-45d9-86d5-3243ccd65497 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:18:59.521575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cd68e-eb5d-45d9-86d5-3243ccd65497 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:18:59.541921 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:18:59.555963 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b64e1f-471f-4b4a-8ed9-facaeaaedddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:18:59.556626 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b64e1f-471f-4b4a-8ed9-facaeaaedddd None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:18:59.556918 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b64e1f-471f-4b4a-8ed9-facaeaaedddd None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:18:59.557216 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b64e1f-471f-4b4a-8ed9-facaeaaedddd None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:18:59.557866 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b64e1f-471f-4b4a-8ed9-facaeaaedddd None admin] RBAC: Adding query filter params (type=image) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:18:59.558171 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b64e1f-471f-4b4a-8ed9-facaeaaedddd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:18:59.559575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b64e1f-471f-4b4a-8ed9-facaeaaedddd None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:18:59.565108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 507/1007] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:18:59 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:00.865458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c937a1-216b-4cd6-b0a1-543d1e984868 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:00.865978 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c937a1-216b-4cd6-b0a1-543d1e984868 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:00.865978 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c937a1-216b-4cd6-b0a1-543d1e984868 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:00.868035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 501/1008] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:00.872973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dcee3-94ed-4ba0-a298-264541828af0 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:00.873391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dcee3-94ed-4ba0-a298-264541828af0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:00.873588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dcee3-94ed-4ba0-a298-264541828af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:00.891870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b86dcee3-94ed-4ba0-a298-264541828af0 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:00.893076 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b86dcee3-94ed-4ba0-a298-264541828af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:00.959963 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 508/1009] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:00 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:01.230018 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c937a1-216b-4cd6-b0a1-543d1e984868 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:01.255630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:01.307759 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464ef57-8a71-4590-bdc4-d9bdd3e2b8f0 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:01.308081 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464ef57-8a71-4590-bdc4-d9bdd3e2b8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:01.308332 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464ef57-8a71-4590-bdc4-d9bdd3e2b8f0 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:01.310026 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 509/1010] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:19:01.314858 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da90b6-165e-4a74-9543-b1fddf25c63f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:01.315145 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da90b6-165e-4a74-9543-b1fddf25c63f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:01.315427 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da90b6-165e-4a74-9543-b1fddf25c63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:01.334819 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-59da90b6-165e-4a74-9543-b1fddf25c63f None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:01.335450 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c937a1-216b-4cd6-b0a1-543d1e984868 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:01.336033 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-59da90b6-165e-4a74-9543-b1fddf25c63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:01.369854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799f223-b8a2-4b71-8e56-e97dd7e27080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:01.370770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799f223-b8a2-4b71-8e56-e97dd7e27080 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:01.371229 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799f223-b8a2-4b71-8e56-e97dd7e27080 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:01.371498 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799f223-b8a2-4b71-8e56-e97dd7e27080 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:01.372075 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799f223-b8a2-4b71-8e56-e97dd7e27080 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:01.373468 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799f223-b8a2-4b71-8e56-e97dd7e27080 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:01.382283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-c799f223-b8a2-4b71-8e56-e97dd7e27080 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 10:19:01.382283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:01.382283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:19:01.382283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:19:01.382283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:19:01.382283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:19:01.382283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:19:01.382283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:19:01.382283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 10:19:01.382283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:01.382283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:19:01.382283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:01.382283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:01.382283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:01.382283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:01.382283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:01.382283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:01.383821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:01.383821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:01.383821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:01.383821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:01.383821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:01.383821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:01.383821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:19:01.383821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:19:01.383821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:19:01.383821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:19:01.383821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:01.383821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:01.383821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:01.383821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:01.383821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:01.383821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:01.385405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:01.385405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:01.385405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:01.385405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:01.385405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:01.385405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:01.385405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:01.385405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:01.385405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:01.385405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:01.385405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:19:01.385405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:19:01.385405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 10:19:01.385405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:01.385405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 502/1011] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:19:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:01.389599 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799f223-b8a2-4b71-8e56-e97dd7e27080 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:01.409001 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 510/1012] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:01 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:19:01.486003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:01.497803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799f223-b8a2-4b71-8e56-e97dd7e27080 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:01.532667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94341c1-8272-413e-8889-459827f0eef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:01.533308 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94341c1-8272-413e-8889-459827f0eef0 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:01.533586 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94341c1-8272-413e-8889-459827f0eef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:01.533858 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94341c1-8272-413e-8889-459827f0eef0 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:01.534480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94341c1-8272-413e-8889-459827f0eef0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:01.534781 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94341c1-8272-413e-8889-459827f0eef0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:01.536133 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94341c1-8272-413e-8889-459827f0eef0 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:01.541722 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 503/1013] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 19 10:19:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:01.547335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da90b6-165e-4a74-9543-b1fddf25c63f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:01.593026 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:01.651579 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da90b6-165e-4a74-9543-b1fddf25c63f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:01.683288 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94341c1-8272-413e-8889-459827f0eef0 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:01.686479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd56fa-f2df-4c5c-bf92-e1a3dcbb143c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:01.687096 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd56fa-f2df-4c5c-bf92-e1a3dcbb143c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:01.687370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd56fa-f2df-4c5c-bf92-e1a3dcbb143c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:01.687660 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd56fa-f2df-4c5c-bf92-e1a3dcbb143c None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:01.688276 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd56fa-f2df-4c5c-bf92-e1a3dcbb143c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:01.688578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd56fa-f2df-4c5c-bf92-e1a3dcbb143c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:01.689951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd56fa-f2df-4c5c-bf92-e1a3dcbb143c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:01.696187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 511/1014] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Sep 19 10:19:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:01.701504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd56fa-f2df-4c5c-bf92-e1a3dcbb143c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:01.784681 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94341c1-8272-413e-8889-459827f0eef0 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:01.804754 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd56fa-f2df-4c5c-bf92-e1a3dcbb143c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:01.814684 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:01.820782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1060d-14f7-45cd-b306-a1ce9a4c0414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:01.821497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1060d-14f7-45cd-b306-a1ce9a4c0414 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:01.821773 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1060d-14f7-45cd-b306-a1ce9a4c0414 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:01.822043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1060d-14f7-45cd-b306-a1ce9a4c0414 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:01.827185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1060d-14f7-45cd-b306-a1ce9a4c0414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:01.827185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1060d-14f7-45cd-b306-a1ce9a4c0414 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:01.827185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 504/1015] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:19:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:01.827947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed1923-d3db-41a3-9862-92ceda64f70e None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:01.827947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed1923-d3db-41a3-9862-92ceda64f70e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:01.827947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed1923-d3db-41a3-9862-92ceda64f70e None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:01.829052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 505/1016] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:01.831906 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed1923-d3db-41a3-9862-92ceda64f70e None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:01.843240 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8e0a1-7043-4e40-ab77-34ff8966ad03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:01.844078 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8e0a1-7043-4e40-ab77-34ff8966ad03 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:01.844615 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8e0a1-7043-4e40-ab77-34ff8966ad03 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:01.844897 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8e0a1-7043-4e40-ab77-34ff8966ad03 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:01.845612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8e0a1-7043-4e40-ab77-34ff8966ad03 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:01.847011 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8e0a1-7043-4e40-ab77-34ff8966ad03 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:01.854075 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:01.905916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 512/1017] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:19:01 2022] POST /identity/v3/endpoints => generated 334 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:19:01.905916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b5a39-7111-4a39-95bf-22d33f63187f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:01.905916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b5a39-7111-4a39-95bf-22d33f63187f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:01.905916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b5a39-7111-4a39-95bf-22d33f63187f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:01.927354 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f48b5a39-7111-4a39-95bf-22d33f63187f None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:01.928594 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f48b5a39-7111-4a39-95bf-22d33f63187f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:01.945203 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed1923-d3db-41a3-9862-92ceda64f70e None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:01.993072 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54603b1a-55b4-4e4d-ad1c-ecc6495a93a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:01.994281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54603b1a-55b4-4e4d-ad1c-ecc6495a93a7 None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:01.994693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54603b1a-55b4-4e4d-ad1c-ecc6495a93a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:01.994965 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54603b1a-55b4-4e4d-ad1c-ecc6495a93a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:01.995882 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54603b1a-55b4-4e4d-ad1c-ecc6495a93a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:01.998151 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54603b1a-55b4-4e4d-ad1c-ecc6495a93a7 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:02.017248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 513/1018] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:01 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:02.068312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 506/1019] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 19 10:19:01 2022] POST /identity/v3/projects => generated 311 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:19:02.273315 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b5a39-7111-4a39-95bf-22d33f63187f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:02.368520 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:02.375380 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b5a39-7111-4a39-95bf-22d33f63187f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:02.408996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305d1a2-bce0-44f7-b1f8-9462ab958beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:02.410090 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305d1a2-bce0-44f7-b1f8-9462ab958beb None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:02.410908 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305d1a2-bce0-44f7-b1f8-9462ab958beb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:02.411189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305d1a2-bce0-44f7-b1f8-9462ab958beb None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:02.411897 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305d1a2-bce0-44f7-b1f8-9462ab958beb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:02.413649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305d1a2-bce0-44f7-b1f8-9462ab958beb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:02.472380 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 514/1020] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 19 10:19:02 2022] POST /identity/v3/services => generated 242 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:19:04.178991 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f08ff6-f029-4e4b-90d0-baee797f31e1 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:04.179531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f08ff6-f029-4e4b-90d0-baee797f31e1 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:04.179531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f08ff6-f029-4e4b-90d0-baee797f31e1 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:04.181101 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 507/1021] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:04.187117 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23b18f-c228-42fa-b529-52c0ad9669f8 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:04.187668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23b18f-c228-42fa-b529-52c0ad9669f8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:04.187668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23b18f-c228-42fa-b529-52c0ad9669f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:04.206952 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-0b23b18f-c228-42fa-b529-52c0ad9669f8 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:04.208147 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0b23b18f-c228-42fa-b529-52c0ad9669f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:04.283917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 515/1022] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:04.296563 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f8873-f064-4cd8-bc2c-a6fb68d0a74b None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:04.296563 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f8873-f064-4cd8-bc2c-a6fb68d0a74b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:04.296563 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f8873-f064-4cd8-bc2c-a6fb68d0a74b None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:04.296563 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 508/1023] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:04.297360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec61ab-a4ca-4c3c-8333-0355fcd0c3df None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:04.297657 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec61ab-a4ca-4c3c-8333-0355fcd0c3df None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:04.297938 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec61ab-a4ca-4c3c-8333-0355fcd0c3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:04.316569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e9ec61ab-a4ca-4c3c-8333-0355fcd0c3df None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:04.317771 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e9ec61ab-a4ca-4c3c-8333-0355fcd0c3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:04.385000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 516/1024] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:04.554097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f8873-f064-4cd8-bc2c-a6fb68d0a74b None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:04.619911 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:04.626008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8223f-feae-415a-ad18-da4712e0bcec None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:04.626302 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8223f-feae-415a-ad18-da4712e0bcec None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:04.626586 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8223f-feae-415a-ad18-da4712e0bcec None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:04.628170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 517/1025] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:19:04.632996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d3340-cc35-4143-a4ff-4c8c6b4676c1 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:04.636773 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d3340-cc35-4143-a4ff-4c8c6b4676c1 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:04.636773 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d3340-cc35-4143-a4ff-4c8c6b4676c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:04.652870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e29d3340-cc35-4143-a4ff-4c8c6b4676c1 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:04.654117 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e29d3340-cc35-4143-a4ff-4c8c6b4676c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:04.656180 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f8873-f064-4cd8-bc2c-a6fb68d0a74b None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:04.658144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:04.691440 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4568d-e27d-430d-a086-6f7d47911542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:04.692067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4568d-e27d-430d-a086-6f7d47911542 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:04.692352 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4568d-e27d-430d-a086-6f7d47911542 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:04.692623 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4568d-e27d-430d-a086-6f7d47911542 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:04.703755 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4568d-e27d-430d-a086-6f7d47911542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:19:04.708692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-fbd4568d-e27d-430d-a086-6f7d47911542 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 10:19:04.708692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:04.708692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:04.708692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:04.708692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:04.708692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:04.708692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:04.708692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:04.708692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:04.708692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:04.708692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:04.708692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:04.708692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:19:04.708692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:19:04.708692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:19:04.708692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:19:04.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:19:04.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:19:04.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:19:04.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:19:04.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:04.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:04.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:04.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:04.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:04.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:04.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:04.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:04.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:04.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:04.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:04.709861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:04.711303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:04.711303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:04.711303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:04.711303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:04.711303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:19:04.711303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:19:04.711303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:19:04.711303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:19:04.711303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:19:04.711303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:19:04.711303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 10:19:04.711303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:04.711303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 509/1026] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Sep 19 10:19:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:04.711303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4568d-e27d-430d-a086-6f7d47911542 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:04.725971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 518/1027] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:19:04.726864 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d3340-cc35-4143-a4ff-4c8c6b4676c1 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:04.826928 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4568d-e27d-430d-a086-6f7d47911542 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:04.838618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d3340-cc35-4143-a4ff-4c8c6b4676c1 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:04.855439 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:04.863723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d245e-b875-4d9e-be94-cb57c025d24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:04.864528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d245e-b875-4d9e-be94-cb57c025d24a None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:04.865709 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d245e-b875-4d9e-be94-cb57c025d24a None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:04.865709 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d245e-b875-4d9e-be94-cb57c025d24a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:04.866937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d245e-b875-4d9e-be94-cb57c025d24a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:04.868436 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d245e-b875-4d9e-be94-cb57c025d24a None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:04.870047 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 510/1028] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:19:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:04.875968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d245e-b875-4d9e-be94-cb57c025d24a None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:04.881161 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803ffde-f647-478b-9802-88620f7a022b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:04.881817 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803ffde-f647-478b-9802-88620f7a022b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:04.882141 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803ffde-f647-478b-9802-88620f7a022b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:04.882405 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803ffde-f647-478b-9802-88620f7a022b None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:04.883073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803ffde-f647-478b-9802-88620f7a022b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:04.883380 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803ffde-f647-478b-9802-88620f7a022b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:04.884734 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803ffde-f647-478b-9802-88620f7a022b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:04.893603 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 519/1029] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Sep 19 10:19:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:04.899515 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803ffde-f647-478b-9802-88620f7a022b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:04.902433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:04.981462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d245e-b875-4d9e-be94-cb57c025d24a None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:05.003356 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803ffde-f647-478b-9802-88620f7a022b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:05.015954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb40fc-4603-4423-951f-56ace7045409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:05.016558 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb40fc-4603-4423-951f-56ace7045409 None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:05.016836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb40fc-4603-4423-951f-56ace7045409 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:05.017136 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb40fc-4603-4423-951f-56ace7045409 None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:05.017778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb40fc-4603-4423-951f-56ace7045409 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:05.019216 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb40fc-4603-4423-951f-56ace7045409 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:05.036460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5bd03-5ab8-402f-954e-a501619aef88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:05.037096 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5bd03-5ab8-402f-954e-a501619aef88 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:05.037384 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5bd03-5ab8-402f-954e-a501619aef88 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:05.042780 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5bd03-5ab8-402f-954e-a501619aef88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:05.049498 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5bd03-5ab8-402f-954e-a501619aef88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:05.050917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5bd03-5ab8-402f-954e-a501619aef88 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:05.065031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-93a5bd03-5ab8-402f-954e-a501619aef88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:19:05.065031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:05.065031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:05.065031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:05.065031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:05.065031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:05.065031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:05.065031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:05.065031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:05.065031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:05.065031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:05.065031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:05.065031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:19:05.065031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:19:05.065031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:19:05.065031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:19:05.070272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:05.070272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:05.070272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:05.070272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:05.070272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:05.070272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:05.070272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:05.070272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:05.070272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:05.070272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:05.070272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:05.070272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:05.070272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:05.070272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:05.070272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:05.070272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:05.070272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:19:05.071370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:19:05.071370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:19:05.071370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:19:05.071370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 520/1030] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:19:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:19:05.071370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5bd03-5ab8-402f-954e-a501619aef88 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:05.129183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 511/1031] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 19 10:19:04 2022] POST /identity/v3/users => generated 285 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:19:05.129183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb40fc-4603-4423-951f-56ace7045409 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:05.129473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:05.173164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5bd03-5ab8-402f-954e-a501619aef88 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:05.176562 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:05.211647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d0ab2-9fba-442b-b8e1-d3da85a7765f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:05.212298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d0ab2-9fba-442b-b8e1-d3da85a7765f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:05.212582 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d0ab2-9fba-442b-b8e1-d3da85a7765f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:05.212862 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d0ab2-9fba-442b-b8e1-d3da85a7765f None admin] PATH_INFO: `/v3/services/image` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:05.213508 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d0ab2-9fba-442b-b8e1-d3da85a7765f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:05.214906 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d0ab2-9fba-442b-b8e1-d3da85a7765f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:05.224732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-016d0ab2-9fba-442b-b8e1-d3da85a7765f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 10:19:05.224732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:05.224732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:19:05.224732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:19:05.224732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:19:05.224732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:19:05.224732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:19:05.224732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:19:05.224732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 10:19:05.224732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:19:05.224732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:19:05.224732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:19:05.224732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:05.224732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:05.224732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:05.224732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:05.224732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:05.226728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:05.226728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:05.226728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:05.226728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:05.226728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:05.226728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:05.226728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:19:05.226728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:19:05.226728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:19:05.226728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:19:05.226728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:05.226728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:05.226728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:05.226728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:05.226728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:05.226728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:05.227605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:05.227605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:05.227605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:05.227605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:05.227605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:05.227605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:05.227605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:05.227605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:05.227605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:05.227605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:05.227605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:19:05.227605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:19:05.227605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 10:19:05.227605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:19:05.227605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 521/1032] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Sep 19 10:19:05 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:05.232451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb40fc-4603-4423-951f-56ace7045409 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:05.233109 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d0ab2-9fba-442b-b8e1-d3da85a7765f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:05.270115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cda26e-2e67-4683-9e1b-615cf0e18905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:05.270766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cda26e-2e67-4683-9e1b-615cf0e18905 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:05.271020 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cda26e-2e67-4683-9e1b-615cf0e18905 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:05.271295 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cda26e-2e67-4683-9e1b-615cf0e18905 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:05.272215 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cda26e-2e67-4683-9e1b-615cf0e18905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:05.273037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cda26e-2e67-4683-9e1b-615cf0e18905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:05.274471 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cda26e-2e67-4683-9e1b-615cf0e18905 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:05.287120 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 512/1033] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:05.293457 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cda26e-2e67-4683-9e1b-615cf0e18905 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:05.348479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d0ab2-9fba-442b-b8e1-d3da85a7765f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:05.369910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:05.388183 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05482cf6-c9c8-4429-8920-0cab646a62f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:05.388717 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05482cf6-c9c8-4429-8920-0cab646a62f6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:05.389192 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05482cf6-c9c8-4429-8920-0cab646a62f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:05.389470 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05482cf6-c9c8-4429-8920-0cab646a62f6 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:05.390513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05482cf6-c9c8-4429-8920-0cab646a62f6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:05.390825 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05482cf6-c9c8-4429-8920-0cab646a62f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:05.392195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05482cf6-c9c8-4429-8920-0cab646a62f6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:05.398609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 522/1034] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:19:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:05.404290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05482cf6-c9c8-4429-8920-0cab646a62f6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:05.414274 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cda26e-2e67-4683-9e1b-615cf0e18905 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:05.431293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:05.457106 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3e689-a139-48f0-9364-652272bda63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:05.458167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3e689-a139-48f0-9364-652272bda63f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:05.458487 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3e689-a139-48f0-9364-652272bda63f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:05.458726 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3e689-a139-48f0-9364-652272bda63f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:05.468784 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3e689-a139-48f0-9364-652272bda63f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:05.470977 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3e689-a139-48f0-9364-652272bda63f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:05.483550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-32e3e689-a139-48f0-9364-652272bda63f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 10:19:05.483550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:05.483550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:05.483550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:05.483550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:05.483550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:05.483550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:05.483550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:05.483550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:05.483550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:05.483550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:05.483550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:05.483550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:19:05.483550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:19:05.483550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:19:05.483550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:19:05.484574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:05.484574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:05.484574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:19:05.484574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:19:05.484574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:19:05.484574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:19:05.484574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:05.484574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:05.484574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:05.484574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:05.484574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:05.484574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:05.484574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:05.484574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:05.484574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:05.484574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:05.484574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:05.484574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:05.485960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:05.485960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:05.485960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:19:05.485960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:19:05.485960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:19:05.485960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:19:05.485960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:19:05.485960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:19:05.485960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 10:19:05.485960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:05.489224 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 513/1035] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Sep 19 10:19:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:05.491118 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3e689-a139-48f0-9364-652272bda63f None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:05.509733 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05482cf6-c9c8-4429-8920-0cab646a62f6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:05.545877 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd50c53-6d5f-4f2d-b2e8-987b3711e41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:05.546490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd50c53-6d5f-4f2d-b2e8-987b3711e41a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:05.546765 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd50c53-6d5f-4f2d-b2e8-987b3711e41a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:05.547035 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd50c53-6d5f-4f2d-b2e8-987b3711e41a None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:05.547668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd50c53-6d5f-4f2d-b2e8-987b3711e41a None admin] RBAC: Adding query filter params (type=image) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:05.547973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd50c53-6d5f-4f2d-b2e8-987b3711e41a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:05.549861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd50c53-6d5f-4f2d-b2e8-987b3711e41a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:05.555448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 523/1036] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:19:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:05.560913 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd50c53-6d5f-4f2d-b2e8-987b3711e41a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:05.598377 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3e689-a139-48f0-9364-652272bda63f None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:05.608253 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:05.635000 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f7ec6-f78b-4de0-8927-a6a525c0f958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:05.635626 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f7ec6-f78b-4de0-8927-a6a525c0f958 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:05.635904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f7ec6-f78b-4de0-8927-a6a525c0f958 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:05.636179 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f7ec6-f78b-4de0-8927-a6a525c0f958 None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:05.636905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f7ec6-f78b-4de0-8927-a6a525c0f958 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:05.637240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f7ec6-f78b-4de0-8927-a6a525c0f958 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:05.638736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f7ec6-f78b-4de0-8927-a6a525c0f958 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:05.657227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 514/1037] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Sep 19 10:19:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:05.662782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f7ec6-f78b-4de0-8927-a6a525c0f958 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:05.672730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd50c53-6d5f-4f2d-b2e8-987b3711e41a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:05.692039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:05.709477 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c65868-6e31-4acb-822d-54cf69620e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:05.710776 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c65868-6e31-4acb-822d-54cf69620e0c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:05.711057 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c65868-6e31-4acb-822d-54cf69620e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:05.711329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c65868-6e31-4acb-822d-54cf69620e0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:05.712029 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c65868-6e31-4acb-822d-54cf69620e0c None admin] RBAC: Adding query filter params (interface=public, service_id=92ad5c4e86cf474487624f01c872db94, region_id=RegionOne) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:05.712339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c65868-6e31-4acb-822d-54cf69620e0c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:05.713712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c65868-6e31-4acb-822d-54cf69620e0c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:05.719260 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 524/1038] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Sep 19 10:19:05 2022] GET /identity/v3/endpoints?service_id=92ad5c4e86cf474487624f01c872db94&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:05.724909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c65868-6e31-4acb-822d-54cf69620e0c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:05.767128 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f7ec6-f78b-4de0-8927-a6a525c0f958 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:05.802155 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551fe19-4886-43f6-a72a-22f54ae4fd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:05.802768 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551fe19-4886-43f6-a72a-22f54ae4fd0e None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:05.803043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551fe19-4886-43f6-a72a-22f54ae4fd0e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:05.803323 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551fe19-4886-43f6-a72a-22f54ae4fd0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:05.833640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c65868-6e31-4acb-822d-54cf69620e0c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:05.844258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:05.870397 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab707ddb-d80a-45da-8cf1-e76c9d9678a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:05.871474 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab707ddb-d80a-45da-8cf1-e76c9d9678a6 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:05.872082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab707ddb-d80a-45da-8cf1-e76c9d9678a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:05.872455 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab707ddb-d80a-45da-8cf1-e76c9d9678a6 None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:05.873088 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab707ddb-d80a-45da-8cf1-e76c9d9678a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:05.874706 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab707ddb-d80a-45da-8cf1-e76c9d9678a6 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:05.882651 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 525/1039] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Sep 19 10:19:05 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:19:05.882871 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551fe19-4886-43f6-a72a-22f54ae4fd0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:05.884384 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551fe19-4886-43f6-a72a-22f54ae4fd0e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:05.917656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:05.995684 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 515/1040] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:19:05 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 10:19:06.001186 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab707ddb-d80a-45da-8cf1-e76c9d9678a6 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:06.111591 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab707ddb-d80a-45da-8cf1-e76c9d9678a6 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:06.144718 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbcc3a-c819-4ca8-8890-81c3a55f8d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:06.145368 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbcc3a-c819-4ca8-8890-81c3a55f8d0c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:06.145646 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbcc3a-c819-4ca8-8890-81c3a55f8d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:06.145916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbcc3a-c819-4ca8-8890-81c3a55f8d0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:06.157113 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbcc3a-c819-4ca8-8890-81c3a55f8d0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:06.158532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbcc3a-c819-4ca8-8890-81c3a55f8d0c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:06.173032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-3bfbcc3a-c819-4ca8-8890-81c3a55f8d0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:19:06.173032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:06.173032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:06.173032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:06.173032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:06.173032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:06.173032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:06.173032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:06.173032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:06.173032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:06.173032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:06.173032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:06.173032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:19:06.173032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:19:06.173032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:19:06.173032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:19:06.173996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:06.173996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:06.173996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:06.173996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:06.173996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:06.173996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:06.173996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:06.173996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:06.173996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:06.173996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:06.173996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:06.173996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:06.173996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:06.173996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:06.173996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:06.173996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:06.173996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:19:06.174990 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:19:06.174990 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:19:06.174990 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:19:06.174990 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 526/1041] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:19:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:06.180021 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551fe19-4886-43f6-a72a-22f54ae4fd0e None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:06.277797 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551fe19-4886-43f6-a72a-22f54ae4fd0e None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:06.306164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:06.312181 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb55d1d-50b2-44d0-8c5c-f854a51d32e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:06.312788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb55d1d-50b2-44d0-8c5c-f854a51d32e5 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:06.313073 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb55d1d-50b2-44d0-8c5c-f854a51d32e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:06.313353 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb55d1d-50b2-44d0-8c5c-f854a51d32e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:06.313984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb55d1d-50b2-44d0-8c5c-f854a51d32e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:06.314279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb55d1d-50b2-44d0-8c5c-f854a51d32e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:06.315681 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb55d1d-50b2-44d0-8c5c-f854a51d32e5 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:06.327468 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 516/1042] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:06.333014 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbcc3a-c819-4ca8-8890-81c3a55f8d0c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:06.361946 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:06.440031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbcc3a-c819-4ca8-8890-81c3a55f8d0c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:06.475243 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0eaeb-b9d4-445d-b796-41a74df079a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:06.475906 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0eaeb-b9d4-445d-b796-41a74df079a3 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:06.476180 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0eaeb-b9d4-445d-b796-41a74df079a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:06.476449 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0eaeb-b9d4-445d-b796-41a74df079a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:06.483011 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0eaeb-b9d4-445d-b796-41a74df079a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:06.484433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0eaeb-b9d4-445d-b796-41a74df079a3 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:06.495334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-e3a0eaeb-b9d4-445d-b796-41a74df079a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:19:06.495334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:06.495334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:06.495334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:06.495334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:06.495334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:06.495334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:06.495334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:06.495334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:06.495334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:06.495334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:06.495334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:06.495334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:19:06.495334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:19:06.495334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:19:06.496224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:19:06.496224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:06.496224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:06.496224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:06.496224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:06.496224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:06.496224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:06.496224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:06.496224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:06.496224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:06.496224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:06.496224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:06.496224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:06.496224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:06.496224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:06.496224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:06.496224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:06.497673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:19:06.497673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:19:06.497673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:19:06.497673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:19:06.497673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:19:06.497673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:19:06.497673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:19:06.497673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:19:06.501252 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 527/1043] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:19:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:06.502234 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb55d1d-50b2-44d0-8c5c-f854a51d32e5 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:06.602059 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb55d1d-50b2-44d0-8c5c-f854a51d32e5 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:06.636003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f07121-9e11-4fc6-99d1-e740d97a5cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:06.636603 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f07121-9e11-4fc6-99d1-e740d97a5cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:06.636877 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f07121-9e11-4fc6-99d1-e740d97a5cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:06.637185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f07121-9e11-4fc6-99d1-e740d97a5cc1 None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:06.637867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f07121-9e11-4fc6-99d1-e740d97a5cc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:06.638168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f07121-9e11-4fc6-99d1-e740d97a5cc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:06.639582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f07121-9e11-4fc6-99d1-e740d97a5cc1 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:06.651749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 517/1044] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:19:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:06.657532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0eaeb-b9d4-445d-b796-41a74df079a3 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:06.744165 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:06.759432 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0eaeb-b9d4-445d-b796-41a74df079a3 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:06.797635 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6550f9-4d51-4f4a-a695-8c8e465a377a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:06.798234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6550f9-4d51-4f4a-a695-8c8e465a377a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:06.798508 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6550f9-4d51-4f4a-a695-8c8e465a377a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:06.798774 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6550f9-4d51-4f4a-a695-8c8e465a377a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:06.799488 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6550f9-4d51-4f4a-a695-8c8e465a377a None admin] RBAC: Adding query filter params (role.id=ccde793bf8ca438a97cf89d53e0a0f1e, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=b4007794a30247c4a318af62d05e3b92) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:06.799790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6550f9-4d51-4f4a-a695-8c8e465a377a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:06.801212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6550f9-4d51-4f4a-a695-8c8e465a377a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:06.813184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 528/1045] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:19:06 2022] GET /identity/v3/role_assignments?user.id=b4007794a30247c4a318af62d05e3b92&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=ccde793bf8ca438a97cf89d53e0a0f1e => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:07.351344 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afab90c-da5c-4cb9-95c8-90d90c743eb5 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:07.351891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afab90c-da5c-4cb9-95c8-90d90c743eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:07.351891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afab90c-da5c-4cb9-95c8-90d90c743eb5 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:07.353265 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 518/1046] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:07.358440 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c13e0-d86a-45a4-bb54-0043db031661 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:07.358853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c13e0-d86a-45a4-bb54-0043db031661 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:07.358995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c13e0-d86a-45a4-bb54-0043db031661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:07.377410 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b59c13e0-d86a-45a4-bb54-0043db031661 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:07.378600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b59c13e0-d86a-45a4-bb54-0043db031661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:07.444412 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 529/1047] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:19:07.715954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afab90c-da5c-4cb9-95c8-90d90c743eb5 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:07.766426 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:07.825759 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afab90c-da5c-4cb9-95c8-90d90c743eb5 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:07.860977 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d99194-88eb-4dff-879a-df28e5e66f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:07.861637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d99194-88eb-4dff-879a-df28e5e66f98 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:07.861918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d99194-88eb-4dff-879a-df28e5e66f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:07.862190 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d99194-88eb-4dff-879a-df28e5e66f98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:07.873266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d99194-88eb-4dff-879a-df28e5e66f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:19:07.878166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-50d99194-88eb-4dff-879a-df28e5e66f98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:19:07.878166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:07.878166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:07.878166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:07.878166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:07.878166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:07.878166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:07.878166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:07.878166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:07.878166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:07.878166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:07.878166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:07.878166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:19:07.878166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:19:07.878166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:19:07.878166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:19:07.879093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:19:07.879093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:19:07.879093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:19:07.879093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:19:07.879093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:07.879093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:07.879093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:07.879093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:07.879093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:07.879093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:07.879093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:07.879093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:07.879093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:07.879093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:07.879093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:07.879093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:07.879947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:07.879947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:07.879947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:07.879947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:07.879947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:19:07.879947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:19:07.879947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:19:07.879947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:19:07.879947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:19:07.879947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:19:07.879947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:19:07.879947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:07.879947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 519/1048] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:19:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:07.886715 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c13e0-d86a-45a4-bb54-0043db031661 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:07.972187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:07.992928 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c13e0-d86a-45a4-bb54-0043db031661 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:08.027573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb9f91-995a-4a0b-a4a2-6a16e9191e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:08.028182 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb9f91-995a-4a0b-a4a2-6a16e9191e24 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:08.028458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb9f91-995a-4a0b-a4a2-6a16e9191e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:08.028732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb9f91-995a-4a0b-a4a2-6a16e9191e24 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:08.029424 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb9f91-995a-4a0b-a4a2-6a16e9191e24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:08.029731 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb9f91-995a-4a0b-a4a2-6a16e9191e24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:08.031082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb9f91-995a-4a0b-a4a2-6a16e9191e24 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:08.044869 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 530/1049] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:19:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:08.045916 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d99194-88eb-4dff-879a-df28e5e66f98 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:08.063016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e49f29-5909-4bb4-8281-35d01bb4c588 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:08.063307 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e49f29-5909-4bb4-8281-35d01bb4c588 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:08.063589 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e49f29-5909-4bb4-8281-35d01bb4c588 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:08.064925 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 531/1050] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:19:08.069794 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f12279-17a7-48b3-9902-f91f10137dcf None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:08.070084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f12279-17a7-48b3-9902-f91f10137dcf None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:08.070368 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f12279-17a7-48b3-9902-f91f10137dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:08.089119 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-26f12279-17a7-48b3-9902-f91f10137dcf None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:08.090312 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-26f12279-17a7-48b3-9902-f91f10137dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:08.152676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d99194-88eb-4dff-879a-df28e5e66f98 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:08.160894 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 532/1051] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:19:08.164307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:08.188119 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2703e-faa0-4b45-9da8-8c02bae89aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:08.188759 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2703e-faa0-4b45-9da8-8c02bae89aed None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:08.189032 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2703e-faa0-4b45-9da8-8c02bae89aed None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:08.189336 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2703e-faa0-4b45-9da8-8c02bae89aed None admin] PATH_INFO: `/v3/users/da16d6d541664d9e9f250f2aa3aeb21b` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:08.202447 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2703e-faa0-4b45-9da8-8c02bae89aed None admin] RBAC: Authorizing `identity:get_user(user_id=da16d6d541664d9e9f250f2aa3aeb21b)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:08.203879 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2703e-faa0-4b45-9da8-8c02bae89aed None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:08.210742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 520/1052] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:19:08 2022] GET /identity/v3/users/da16d6d541664d9e9f250f2aa3aeb21b => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:08.211784 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f12279-17a7-48b3-9902-f91f10137dcf None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:08.280823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:08.310771 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f12279-17a7-48b3-9902-f91f10137dcf None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:08.343689 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d91289-d8a0-49d3-9fe1-291507d3881b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:08.344297 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d91289-d8a0-49d3-9fe1-291507d3881b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:08.344575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d91289-d8a0-49d3-9fe1-291507d3881b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:08.344850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d91289-d8a0-49d3-9fe1-291507d3881b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:08.411476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d91289-d8a0-49d3-9fe1-291507d3881b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:08.412980 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d91289-d8a0-49d3-9fe1-291507d3881b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:08.433797 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2703e-faa0-4b45-9da8-8c02bae89aed None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:08.484618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:08.517042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 533/1053] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:19:08 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:19:08.523763 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d91289-d8a0-49d3-9fe1-291507d3881b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:08.539152 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2703e-faa0-4b45-9da8-8c02bae89aed None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:08.573538 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29156a7d-53fe-4ee3-868d-9322a85fd1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:08.574266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29156a7d-53fe-4ee3-868d-9322a85fd1d2 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:08.574570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29156a7d-53fe-4ee3-868d-9322a85fd1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:08.574847 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29156a7d-53fe-4ee3-868d-9322a85fd1d2 None admin] PATH_INFO: `/v3/services/image` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:08.575422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29156a7d-53fe-4ee3-868d-9322a85fd1d2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:08.576782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29156a7d-53fe-4ee3-868d-9322a85fd1d2 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:08.586958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-29156a7d-53fe-4ee3-868d-9322a85fd1d2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 10:19:08.586958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:08.586958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:19:08.586958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:19:08.586958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:19:08.586958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:19:08.586958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:19:08.586958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:19:08.586958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 10:19:08.586958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:08.586958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:19:08.586958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:08.586958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:08.586958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:08.586958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:08.586958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:08.586958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:08.588067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:08.588067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:08.588067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:08.588067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:08.588067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:08.588067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:08.588067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:19:08.588067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:19:08.588067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:19:08.588067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:19:08.588067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:08.588067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:08.588067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:08.588067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:08.588067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:08.588067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:08.588937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:08.588937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:08.588937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:08.588937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:08.588937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:08.588937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:08.588937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:08.588937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:08.588937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:08.588937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:08.588937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:19:08.588937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:19:08.588937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 10:19:08.588937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:08.588937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 521/1054] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Sep 19 10:19:08 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:19:08.593872 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29156a7d-53fe-4ee3-868d-9322a85fd1d2 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:08.629321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d91289-d8a0-49d3-9fe1-291507d3881b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:08.629543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:08.665210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9629c6-38e2-49a1-8c40-2a55fadd99aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:08.665840 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9629c6-38e2-49a1-8c40-2a55fadd99aa None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:08.666113 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9629c6-38e2-49a1-8c40-2a55fadd99aa None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:08.666384 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9629c6-38e2-49a1-8c40-2a55fadd99aa None admin] PATH_INFO: `/v3/projects/45a08b39cc00409597972684860476e0` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:08.667494 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9629c6-38e2-49a1-8c40-2a55fadd99aa None admin] RBAC: Authorizing `identity:get_project(project_id=45a08b39cc00409597972684860476e0)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:08.668918 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9629c6-38e2-49a1-8c40-2a55fadd99aa None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:08.675496 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 534/1055] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:19:08 2022] GET /identity/v3/projects/45a08b39cc00409597972684860476e0 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:08.676571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9629c6-38e2-49a1-8c40-2a55fadd99aa None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:08.703146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29156a7d-53fe-4ee3-868d-9322a85fd1d2 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:08.720710 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:08.739288 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48452004-17d6-4723-91ce-6125a0dfcad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:08.739894 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48452004-17d6-4723-91ce-6125a0dfcad1 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:08.740195 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48452004-17d6-4723-91ce-6125a0dfcad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:08.740446 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48452004-17d6-4723-91ce-6125a0dfcad1 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:08.741133 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48452004-17d6-4723-91ce-6125a0dfcad1 None admin] RBAC: Adding query filter params (name=image) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:08.741440 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48452004-17d6-4723-91ce-6125a0dfcad1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:08.742810 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48452004-17d6-4723-91ce-6125a0dfcad1 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:08.748361 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 522/1056] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:19:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:08.754008 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48452004-17d6-4723-91ce-6125a0dfcad1 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:08.781331 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9629c6-38e2-49a1-8c40-2a55fadd99aa None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:08.825197 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787390ba-799b-406a-9685-5e48be59a1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:08.826082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787390ba-799b-406a-9685-5e48be59a1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:08.826082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787390ba-799b-406a-9685-5e48be59a1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:08.826260 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787390ba-799b-406a-9685-5e48be59a1f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:08.827025 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787390ba-799b-406a-9685-5e48be59a1f4 None admin] RBAC: Adding query filter params (role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.project.id=45a08b39cc00409597972684860476e0, user.id=da16d6d541664d9e9f250f2aa3aeb21b) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:08.827324 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787390ba-799b-406a-9685-5e48be59a1f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:08.828791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787390ba-799b-406a-9685-5e48be59a1f4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:08.845390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 535/1057] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:19:08 2022] GET /identity/v3/role_assignments?user.id=da16d6d541664d9e9f250f2aa3aeb21b&scope.project.id=45a08b39cc00409597972684860476e0&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:08.871039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:08.877488 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48452004-17d6-4723-91ce-6125a0dfcad1 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:08.915063 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56aef7f-ce61-4817-9a93-60eaeb297d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:08.915687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56aef7f-ce61-4817-9a93-60eaeb297d20 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:08.915964 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56aef7f-ce61-4817-9a93-60eaeb297d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:08.916244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56aef7f-ce61-4817-9a93-60eaeb297d20 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:08.916886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56aef7f-ce61-4817-9a93-60eaeb297d20 None admin] RBAC: Adding query filter params (type=image) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:08.917221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56aef7f-ce61-4817-9a93-60eaeb297d20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:08.918633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56aef7f-ce61-4817-9a93-60eaeb297d20 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:08.924206 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 523/1058] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:19:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:08.930796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787390ba-799b-406a-9685-5e48be59a1f4 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:08.951145 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb4f90-b4bc-489c-8c01-1c1df1fa051d None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:08.951429 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb4f90-b4bc-489c-8c01-1c1df1fa051d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:08.951700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb4f90-b4bc-489c-8c01-1c1df1fa051d None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:08.954284 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 524/1059] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:19:08.958959 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c2af5-2598-4cb6-8f2f-0f156c944d75 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:08.959492 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c2af5-2598-4cb6-8f2f-0f156c944d75 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:08.959803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c2af5-2598-4cb6-8f2f-0f156c944d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:08.979090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-d37c2af5-2598-4cb6-8f2f-0f156c944d75 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:08.980322 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d37c2af5-2598-4cb6-8f2f-0f156c944d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:09.045263 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787390ba-799b-406a-9685-5e48be59a1f4 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:09.056925 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 525/1060] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:19:09.057463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:09.082195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f2a0f-529a-4d50-bb29-bc45d2274a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:09.082372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f2a0f-529a-4d50-bb29-bc45d2274a0f None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:09.082593 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f2a0f-529a-4d50-bb29-bc45d2274a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:09.082831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f2a0f-529a-4d50-bb29-bc45d2274a0f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:09.083547 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f2a0f-529a-4d50-bb29-bc45d2274a0f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:09.084917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f2a0f-529a-4d50-bb29-bc45d2274a0f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:09.163572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 536/1061] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:19:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:19:09.326742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c2af5-2598-4cb6-8f2f-0f156c944d75 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:09.421083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:09.430868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c2af5-2598-4cb6-8f2f-0f156c944d75 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:09.471492 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b66d5-59bd-464a-af4f-381ba747c08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:09.472126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b66d5-59bd-464a-af4f-381ba747c08e None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:09.472375 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b66d5-59bd-464a-af4f-381ba747c08e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:09.472641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b66d5-59bd-464a-af4f-381ba747c08e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:09.482968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b66d5-59bd-464a-af4f-381ba747c08e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:19:09.487690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-e97b66d5-59bd-464a-af4f-381ba747c08e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 10:19:09.487690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:09.487690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:09.487690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:09.487690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:09.487690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:09.487690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:09.487690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:09.487690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:09.487690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:09.487690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:09.487690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:09.487690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:19:09.487690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:19:09.487690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:19:09.487690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:19:09.488641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:19:09.488641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:19:09.488641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:19:09.488641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:19:09.488641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:09.488641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:09.488641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:09.488641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:09.488641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:09.488641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:09.488641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:09.488641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:09.488641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:09.488641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:09.488641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:09.488641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:09.489943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:09.489943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:09.489943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:09.489943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:09.489943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:19:09.489943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:19:09.489943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:19:09.489943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:19:09.489943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:19:09.489943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:19:09.489943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 10:19:09.489943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:09.489943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 526/1062] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Sep 19 10:19:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:09.495097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f2a0f-529a-4d50-bb29-bc45d2274a0f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:09.595972 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f2a0f-529a-4d50-bb29-bc45d2274a0f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:09.629179 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e8b0e-8977-4d12-889e-ae5b84e0a233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:09.629785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e8b0e-8977-4d12-889e-ae5b84e0a233 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:09.630060 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e8b0e-8977-4d12-889e-ae5b84e0a233 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:09.630327 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e8b0e-8977-4d12-889e-ae5b84e0a233 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:09.630978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e8b0e-8977-4d12-889e-ae5b84e0a233 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:09.631277 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e8b0e-8977-4d12-889e-ae5b84e0a233 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:09.633369 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e8b0e-8977-4d12-889e-ae5b84e0a233 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:09.645991 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 537/1063] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Sep 19 10:19:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:09.647828 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b66d5-59bd-464a-af4f-381ba747c08e None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:09.754365 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b66d5-59bd-464a-af4f-381ba747c08e None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:09.788160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90792ae-0202-4b5c-9106-52177b516cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:09.789019 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90792ae-0202-4b5c-9106-52177b516cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:09.789520 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90792ae-0202-4b5c-9106-52177b516cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:09.789976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90792ae-0202-4b5c-9106-52177b516cd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:09.800299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90792ae-0202-4b5c-9106-52177b516cd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:09.801949 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90792ae-0202-4b5c-9106-52177b516cd2 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:09.815474 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:09.828199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-e90792ae-0202-4b5c-9106-52177b516cd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:19:09.828199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:09.828199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:09.828199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:09.828199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:09.828199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:09.828199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:09.828199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:09.828199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:09.828199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:09.828199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:09.828199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:09.828199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:19:09.828199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:19:09.828199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:19:09.828199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:19:09.834688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:09.834688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:09.834688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:09.834688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:09.834688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:09.834688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:09.834688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:09.834688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:09.834688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:09.834688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:09.834688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:09.834688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:09.834688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:09.834688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:09.834688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:09.834688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:09.834688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:19:09.835668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:19:09.835668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:19:09.835668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:09.835668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 527/1064] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:19:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:09.835902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e8b0e-8977-4d12-889e-ae5b84e0a233 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:09.861255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:09.938004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e8b0e-8977-4d12-889e-ae5b84e0a233 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:09.971692 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f359c-8bfa-4b93-a60d-7f83492f994a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:09.972292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f359c-8bfa-4b93-a60d-7f83492f994a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:09.972566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f359c-8bfa-4b93-a60d-7f83492f994a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:09.972833 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f359c-8bfa-4b93-a60d-7f83492f994a None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:09.973495 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f359c-8bfa-4b93-a60d-7f83492f994a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:09.973795 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f359c-8bfa-4b93-a60d-7f83492f994a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:09.975129 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f359c-8bfa-4b93-a60d-7f83492f994a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:09.987098 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 538/1065] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:09.993179 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90792ae-0202-4b5c-9106-52177b516cd2 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:10.097323 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90792ae-0202-4b5c-9106-52177b516cd2 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:10.130970 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb80e2-5dc9-4edf-8e68-403cdc4cd44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:10.131624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb80e2-5dc9-4edf-8e68-403cdc4cd44c None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:10.131901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb80e2-5dc9-4edf-8e68-403cdc4cd44c None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:10.132168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb80e2-5dc9-4edf-8e68-403cdc4cd44c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:10.139483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb80e2-5dc9-4edf-8e68-403cdc4cd44c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:10.140906 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb80e2-5dc9-4edf-8e68-403cdc4cd44c None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:10.152243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-9aeb80e2-5dc9-4edf-8e68-403cdc4cd44c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 10:19:10.152243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:10.152243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:10.152243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:10.152243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:10.152243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:10.152243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:10.152243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:10.152243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:10.152243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:10.152243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:10.152243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:10.152243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:19:10.152243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:19:10.152243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:19:10.152243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:19:10.153497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:10.153497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:10.153497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:19:10.153497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:19:10.153497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:19:10.153497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:19:10.153497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:10.153497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:10.153497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:10.153497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:10.153497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:10.153497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:10.153497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:10.153497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:10.153497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:10.153497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:10.153497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:10.153497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:10.154904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:10.154904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:10.154904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:19:10.154904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:19:10.154904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:19:10.154904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:19:10.154904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:19:10.154904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:19:10.154904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 10:19:10.154904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:10.154904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 528/1066] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Sep 19 10:19:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:10.161242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f359c-8bfa-4b93-a60d-7f83492f994a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:10.244818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:10.262713 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f359c-8bfa-4b93-a60d-7f83492f994a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:10.296342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe622d2-c23f-4170-ae46-7c6917ad34fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:10.297084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe622d2-c23f-4170-ae46-7c6917ad34fa None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:10.297490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe622d2-c23f-4170-ae46-7c6917ad34fa None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:10.297862 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe622d2-c23f-4170-ae46-7c6917ad34fa None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:10.298677 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe622d2-c23f-4170-ae46-7c6917ad34fa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:10.299083 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe622d2-c23f-4170-ae46-7c6917ad34fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:10.300573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe622d2-c23f-4170-ae46-7c6917ad34fa None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:10.323023 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 539/1067] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Sep 19 10:19:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:10.324171 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb80e2-5dc9-4edf-8e68-403cdc4cd44c None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:10.381367 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:10.423516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb80e2-5dc9-4edf-8e68-403cdc4cd44c None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:10.458927 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac117e46-5943-40a3-9dbb-38be5fc48609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:10.459560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac117e46-5943-40a3-9dbb-38be5fc48609 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:10.459863 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac117e46-5943-40a3-9dbb-38be5fc48609 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:10.460136 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac117e46-5943-40a3-9dbb-38be5fc48609 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:10.470649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac117e46-5943-40a3-9dbb-38be5fc48609 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:10.472089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac117e46-5943-40a3-9dbb-38be5fc48609 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:10.486766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-ac117e46-5943-40a3-9dbb-38be5fc48609 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:19:10.486766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:10.486766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:10.486766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:10.486766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:10.486766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:10.486766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:10.486766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:10.486766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:10.486766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:10.486766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:10.486766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:10.486766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:19:10.486766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:19:10.486766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:19:10.486766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:19:10.487787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:10.487787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:10.487787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:10.487787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:10.487787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:10.487787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:10.487787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:10.487787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:10.487787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:10.487787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:10.487787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:10.487787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:10.487787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:10.487787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:10.487787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:10.487787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:10.487787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:19:10.488704 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:19:10.488704 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:19:10.488704 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:10.489598 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 529/1068] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:19:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:10.494242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe622d2-c23f-4170-ae46-7c6917ad34fa None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:10.594471 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe622d2-c23f-4170-ae46-7c6917ad34fa None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:10.628079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75cd50-9a9e-4bc3-b507-fe80826e513c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:10.628681 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75cd50-9a9e-4bc3-b507-fe80826e513c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:10.628963 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75cd50-9a9e-4bc3-b507-fe80826e513c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:10.629270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75cd50-9a9e-4bc3-b507-fe80826e513c None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:10.629903 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75cd50-9a9e-4bc3-b507-fe80826e513c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:10.630208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75cd50-9a9e-4bc3-b507-fe80826e513c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:10.632238 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75cd50-9a9e-4bc3-b507-fe80826e513c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:10.640288 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:10.649020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 540/1069] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:10.653033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac117e46-5943-40a3-9dbb-38be5fc48609 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:10.751431 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac117e46-5943-40a3-9dbb-38be5fc48609 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:10.761979 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:10.785863 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c973523-f992-4b5b-aff7-dc413c0ce214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:10.786496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c973523-f992-4b5b-aff7-dc413c0ce214 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:10.786775 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c973523-f992-4b5b-aff7-dc413c0ce214 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:10.787052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c973523-f992-4b5b-aff7-dc413c0ce214 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:10.795242 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c973523-f992-4b5b-aff7-dc413c0ce214 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:10.796689 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c973523-f992-4b5b-aff7-dc413c0ce214 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:10.808822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-8c973523-f992-4b5b-aff7-dc413c0ce214 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:19:10.808822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:10.808822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:10.808822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:10.808822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:10.808822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:10.808822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:10.808822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:10.808822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:10.808822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:10.808822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:10.808822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:10.808822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:19:10.808822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:19:10.808822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:19:10.809954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:19:10.809954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:10.809954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:10.809954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:10.809954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:10.809954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:10.809954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:10.809954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:10.809954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:10.809954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:10.809954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:10.809954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:10.809954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:10.809954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:10.809954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:10.809954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:10.809954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:10.810892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:19:10.810892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:19:10.810892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:19:10.810892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:19:10.810892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:19:10.810892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:19:10.810892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:19:10.810892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:10.811873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 530/1070] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:19:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:19:10.815925 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75cd50-9a9e-4bc3-b507-fe80826e513c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:10.924618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75cd50-9a9e-4bc3-b507-fe80826e513c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:10.960058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97a8d0-b904-467a-8603-59e042c990b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:10.960693 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97a8d0-b904-467a-8603-59e042c990b5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:10.960976 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97a8d0-b904-467a-8603-59e042c990b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:10.961273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97a8d0-b904-467a-8603-59e042c990b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:10.961963 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97a8d0-b904-467a-8603-59e042c990b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:10.962278 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97a8d0-b904-467a-8603-59e042c990b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:10.963694 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97a8d0-b904-467a-8603-59e042c990b5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:10.981494 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 541/1071] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:19:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:10.982645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c973523-f992-4b5b-aff7-dc413c0ce214 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:11.030257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218d87d-d6fa-4899-99bf-358f772f43c8 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:11.030616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218d87d-d6fa-4899-99bf-358f772f43c8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:11.030822 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218d87d-d6fa-4899-99bf-358f772f43c8 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:11.032287 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 542/1072] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:19:11.036902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ed3cb-ce55-49cb-a2a9-7b1576c0c502 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:11.037211 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ed3cb-ce55-49cb-a2a9-7b1576c0c502 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:11.037486 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ed3cb-ce55-49cb-a2a9-7b1576c0c502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:11.056163 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-053ed3cb-ce55-49cb-a2a9-7b1576c0c502 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:11.057375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-053ed3cb-ce55-49cb-a2a9-7b1576c0c502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:11.088097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c973523-f992-4b5b-aff7-dc413c0ce214 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:11.116805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:11.125426 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8dfe0-ded7-4392-9eca-bc1d2baea500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:11.133240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8dfe0-ded7-4392-9eca-bc1d2baea500 None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:11.133240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8dfe0-ded7-4392-9eca-bc1d2baea500 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:11.133240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8dfe0-ded7-4392-9eca-bc1d2baea500 None admin] PATH_INFO: `/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/b4007794a30247c4a318af62d05e3b92/roles/ccde793bf8ca438a97cf89d53e0a0f1e` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:11.133472 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 543/1073] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:19:11.133545 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8dfe0-ded7-4392-9eca-bc1d2baea500 None admin] RBAC: Authorizing `identity:create_grant(project_id=e45b8fbe79994e7585095328a7ec9260, user_id=b4007794a30247c4a318af62d05e3b92, role_id=ccde793bf8ca438a97cf89d53e0a0f1e)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:11.133545 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8dfe0-ded7-4392-9eca-bc1d2baea500 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:11.231964 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 531/1074] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:19:10 2022] PUT /identity/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/b4007794a30247c4a318af62d05e3b92/roles/ccde793bf8ca438a97cf89d53e0a0f1e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:19:11.353396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac432e76-57ac-4ee0-839e-8eecb58ba11b None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:11.353717 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac432e76-57ac-4ee0-839e-8eecb58ba11b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:11.353955 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac432e76-57ac-4ee0-839e-8eecb58ba11b None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:11.355520 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 544/1075] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:11.361013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4246a8-4e77-4f3a-b3a9-0a5fa7cea287 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:11.361341 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4246a8-4e77-4f3a-b3a9-0a5fa7cea287 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:11.361597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4246a8-4e77-4f3a-b3a9-0a5fa7cea287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:11.381316 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-fe4246a8-4e77-4f3a-b3a9-0a5fa7cea287 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:11.383122 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-fe4246a8-4e77-4f3a-b3a9-0a5fa7cea287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:11.406916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac432e76-57ac-4ee0-839e-8eecb58ba11b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:11.415284 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:11.461104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 532/1076] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:11.520578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac432e76-57ac-4ee0-839e-8eecb58ba11b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:11.554756 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391467c5-6f86-40b1-9e78-e22ee172abe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:11.555369 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391467c5-6f86-40b1-9e78-e22ee172abe7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:11.555645 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391467c5-6f86-40b1-9e78-e22ee172abe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:11.555915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391467c5-6f86-40b1-9e78-e22ee172abe7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:11.566099 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391467c5-6f86-40b1-9e78-e22ee172abe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:19:11.571355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-391467c5-6f86-40b1-9e78-e22ee172abe7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:19:11.571355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:11.571355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:11.571355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:11.571355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:11.571355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:11.571355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:11.571355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:11.571355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:11.571355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:11.571355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:11.571355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:11.571355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:19:11.571355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:19:11.571355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:19:11.571355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:19:11.572333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:19:11.572333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:19:11.572333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:19:11.572333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:19:11.572333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:11.572333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:11.572333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:11.572333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:11.572333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:11.572333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:11.572333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:11.572333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:11.572333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:11.572333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:11.572333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:11.572333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:11.573633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:11.573633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:11.573633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:11.573633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:11.573633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:19:11.573633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:19:11.573633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:19:11.573633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:19:11.573633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:19:11.573633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:19:11.573633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:19:11.573633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:19:11.574514 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 545/1077] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:19:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:19:11.578695 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4246a8-4e77-4f3a-b3a9-0a5fa7cea287 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:11.677391 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4246a8-4e77-4f3a-b3a9-0a5fa7cea287 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:11.712747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0e29e-300b-4fef-b5ab-05723506a205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:11.713379 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0e29e-300b-4fef-b5ab-05723506a205 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:11.713649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0e29e-300b-4fef-b5ab-05723506a205 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:11.713914 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0e29e-300b-4fef-b5ab-05723506a205 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:11.714568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0e29e-300b-4fef-b5ab-05723506a205 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:11.714865 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0e29e-300b-4fef-b5ab-05723506a205 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:11.716213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0e29e-300b-4fef-b5ab-05723506a205 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:11.729104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 533/1078] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:19:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:11.729267 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391467c5-6f86-40b1-9e78-e22ee172abe7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:11.731037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0e29e-300b-4fef-b5ab-05723506a205 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:11.757123 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:11.790366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:11.841838 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391467c5-6f86-40b1-9e78-e22ee172abe7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:11.846694 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0e29e-300b-4fef-b5ab-05723506a205 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:11.876974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1d03e-12a0-40ea-9f34-bc3bd7fab2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:11.877606 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1d03e-12a0-40ea-9f34-bc3bd7fab2fe None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:11.877887 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1d03e-12a0-40ea-9f34-bc3bd7fab2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:11.878163 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1d03e-12a0-40ea-9f34-bc3bd7fab2fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:11.882689 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abfb853-3ccf-40cf-8830-048d1bdb03f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:11.883316 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abfb853-3ccf-40cf-8830-048d1bdb03f6 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:11.883589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abfb853-3ccf-40cf-8830-048d1bdb03f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:11.883856 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abfb853-3ccf-40cf-8830-048d1bdb03f6 None admin] PATH_INFO: `/v3/users/da16d6d541664d9e9f250f2aa3aeb21b` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:11.884976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abfb853-3ccf-40cf-8830-048d1bdb03f6 None admin] RBAC: Authorizing `identity:get_user(user_id=da16d6d541664d9e9f250f2aa3aeb21b)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:11.886429 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abfb853-3ccf-40cf-8830-048d1bdb03f6 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:11.888003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 534/1079] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:19:11 2022] GET /identity/v3/users/da16d6d541664d9e9f250f2aa3aeb21b => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:11.893538 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abfb853-3ccf-40cf-8830-048d1bdb03f6 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:11.958526 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1d03e-12a0-40ea-9f34-bc3bd7fab2fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:11.960497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1d03e-12a0-40ea-9f34-bc3bd7fab2fe None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:11.984637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:11.994090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abfb853-3ccf-40cf-8830-048d1bdb03f6 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:12.011758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:12.027752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd772f03-4f7c-4df0-a669-985b8b6c3c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:12.028380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd772f03-4f7c-4df0-a669-985b8b6c3c92 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:12.028660 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd772f03-4f7c-4df0-a669-985b8b6c3c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:12.028934 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd772f03-4f7c-4df0-a669-985b8b6c3c92 None admin] PATH_INFO: `/v3/projects/45a08b39cc00409597972684860476e0` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:12.030071 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd772f03-4f7c-4df0-a669-985b8b6c3c92 None admin] RBAC: Authorizing `identity:get_project(project_id=45a08b39cc00409597972684860476e0)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:12.031509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd772f03-4f7c-4df0-a669-985b8b6c3c92 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:12.038766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 535/1080] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:19:11 2022] GET /identity/v3/projects/45a08b39cc00409597972684860476e0 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:12.039393 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd772f03-4f7c-4df0-a669-985b8b6c3c92 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:12.069185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 546/1081] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:19:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 10:19:12.071327 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1d03e-12a0-40ea-9f34-bc3bd7fab2fe None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:12.148633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd772f03-4f7c-4df0-a669-985b8b6c3c92 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:12.185535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fa91b-292f-4388-b8e9-3d85fa1b484b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:12.185718 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1d03e-12a0-40ea-9f34-bc3bd7fab2fe None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:12.186196 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fa91b-292f-4388-b8e9-3d85fa1b484b None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:12.186477 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fa91b-292f-4388-b8e9-3d85fa1b484b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:12.186756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fa91b-292f-4388-b8e9-3d85fa1b484b None admin] PATH_INFO: `/v3/projects/45a08b39cc00409597972684860476e0/users/da16d6d541664d9e9f250f2aa3aeb21b/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:12.189348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fa91b-292f-4388-b8e9-3d85fa1b484b None admin] RBAC: Authorizing `identity:create_grant(project_id=45a08b39cc00409597972684860476e0, user_id=da16d6d541664d9e9f250f2aa3aeb21b, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:12.190783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fa91b-292f-4388-b8e9-3d85fa1b484b None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:12.218890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c61852-1339-4da0-a6f1-315fffbaf835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:12.219554 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c61852-1339-4da0-a6f1-315fffbaf835 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:12.219834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c61852-1339-4da0-a6f1-315fffbaf835 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:12.220102 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c61852-1339-4da0-a6f1-315fffbaf835 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:12.230291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c61852-1339-4da0-a6f1-315fffbaf835 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:12.231697 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c61852-1339-4da0-a6f1-315fffbaf835 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:12.246310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-47c61852-1339-4da0-a6f1-315fffbaf835 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:19:12.246310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:12.246310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:12.246310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:12.246310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:12.246310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:12.246310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:12.246310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:12.246310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:12.246310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:12.246310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:12.246310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:12.246310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:19:12.246310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:19:12.246310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:19:12.246310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:19:12.247425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:12.247425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:12.247425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:12.247425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:12.247425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:12.247425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:12.247425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:12.247425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:12.247425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:12.247425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:12.247425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:12.247425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:12.247425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:12.247425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:12.247425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:12.247425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:12.247425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:19:12.248356 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:19:12.248356 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:19:12.248356 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:19:12.248356 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 547/1082] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:19:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:19:12.253308 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c61852-1339-4da0-a6f1-315fffbaf835 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:12.255785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:12.265607 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 536/1083] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:19:12 2022] PUT /identity/v3/projects/45a08b39cc00409597972684860476e0/users/da16d6d541664d9e9f250f2aa3aeb21b/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:19:12.371511 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c61852-1339-4da0-a6f1-315fffbaf835 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:12.408434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984ea5d-01eb-4247-a8ac-03f8034393e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:12.409076 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984ea5d-01eb-4247-a8ac-03f8034393e1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:12.409358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984ea5d-01eb-4247-a8ac-03f8034393e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:12.410824 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984ea5d-01eb-4247-a8ac-03f8034393e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:12.410824 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7984ea5d-01eb-4247-a8ac-03f8034393e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:12.410824 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7984ea5d-01eb-4247-a8ac-03f8034393e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:12.412172 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7984ea5d-01eb-4247-a8ac-03f8034393e1 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:12.433106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 548/1084] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:13.415187 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dfbe7-46f0-4169-9ab0-01867ca2b605 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:13.415744 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dfbe7-46f0-4169-9ab0-01867ca2b605 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:13.415744 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dfbe7-46f0-4169-9ab0-01867ca2b605 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:13.417254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 537/1085] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:13.422322 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c19bf5-a5ec-4267-9890-e0f520c82d7f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:13.422713 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c19bf5-a5ec-4267-9890-e0f520c82d7f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:13.422880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c19bf5-a5ec-4267-9890-e0f520c82d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:13.441605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-78c19bf5-a5ec-4267-9890-e0f520c82d7f None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:13.442761 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-78c19bf5-a5ec-4267-9890-e0f520c82d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:13.497233 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:13.507461 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 549/1086] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:13.776758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dfbe7-46f0-4169-9ab0-01867ca2b605 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:13.796398 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:13.883905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dfbe7-46f0-4169-9ab0-01867ca2b605 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:13.916765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e21f0-1508-41d9-865f-a6b44a7d7c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:13.917428 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e21f0-1508-41d9-865f-a6b44a7d7c51 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:13.917707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e21f0-1508-41d9-865f-a6b44a7d7c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:13.917978 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e21f0-1508-41d9-865f-a6b44a7d7c51 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:13.928158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e21f0-1508-41d9-865f-a6b44a7d7c51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:19:13.932975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-9d8e21f0-1508-41d9-865f-a6b44a7d7c51 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 10:19:13.932975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:13.932975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:13.932975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:13.932975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:13.932975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:13.932975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:13.932975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:13.932975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:13.932975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:13.932975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:13.932975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:13.932975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:19:13.932975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:19:13.932975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:19:13.932975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:19:13.933929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:19:13.933929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:19:13.933929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:19:13.933929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:19:13.933929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:13.933929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:13.933929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:13.933929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:13.933929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:13.933929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:13.933929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:13.933929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:13.933929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:13.933929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:13.933929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:13.933929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:13.934803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:13.934803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:13.934803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:13.934803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:13.934803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:19:13.934803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:19:13.934803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:19:13.934803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:19:13.934803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:19:13.934803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:19:13.934803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 10:19:13.934803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:13.934803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 538/1087] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Sep 19 10:19:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:13.940013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c19bf5-a5ec-4267-9890-e0f520c82d7f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:14.037367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c19bf5-a5ec-4267-9890-e0f520c82d7f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:14.071173 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2f576-73b7-4140-a0a3-c1a24e28fb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:14.071778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2f576-73b7-4140-a0a3-c1a24e28fb1b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:14.072057 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2f576-73b7-4140-a0a3-c1a24e28fb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:14.072332 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2f576-73b7-4140-a0a3-c1a24e28fb1b None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:14.072994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2f576-73b7-4140-a0a3-c1a24e28fb1b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:14.073329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2f576-73b7-4140-a0a3-c1a24e28fb1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:14.074697 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2f576-73b7-4140-a0a3-c1a24e28fb1b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:14.082564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 550/1088] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Sep 19 10:19:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:14.089080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e21f0-1508-41d9-865f-a6b44a7d7c51 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:14.171727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:14.193176 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e21f0-1508-41d9-865f-a6b44a7d7c51 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:14.226669 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad297e-6229-4633-9c3f-e6c46aa322d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:14.227293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad297e-6229-4633-9c3f-e6c46aa322d5 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:14.227572 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad297e-6229-4633-9c3f-e6c46aa322d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:14.227846 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad297e-6229-4633-9c3f-e6c46aa322d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:14.238258 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad297e-6229-4633-9c3f-e6c46aa322d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:14.239685 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad297e-6229-4633-9c3f-e6c46aa322d5 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:14.253542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-34ad297e-6229-4633-9c3f-e6c46aa322d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:19:14.253542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:14.253542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:14.253542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:14.253542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:14.253542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:14.253542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:14.253542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:14.253542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:14.253542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:14.253542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:14.253542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:14.253542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:19:14.253542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:19:14.253542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:19:14.253542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:19:14.254653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:14.254653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:14.254653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:14.254653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:14.254653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:14.254653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:14.254653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:14.254653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:14.254653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:14.254653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:14.254653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:14.254653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:14.254653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:14.254653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:14.254653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:14.254653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:14.254653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:19:14.255889 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:19:14.255889 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:19:14.255889 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:14.255889 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 539/1089] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:19:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:14.260322 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2f576-73b7-4140-a0a3-c1a24e28fb1b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:14.317140 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:14.363060 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2f576-73b7-4140-a0a3-c1a24e28fb1b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:14.394651 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ece2d-735b-44bb-a8d2-0507502f069f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:14.395250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ece2d-735b-44bb-a8d2-0507502f069f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:14.395524 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ece2d-735b-44bb-a8d2-0507502f069f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:14.395789 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ece2d-735b-44bb-a8d2-0507502f069f None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:14.396678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ece2d-735b-44bb-a8d2-0507502f069f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:14.397540 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ece2d-735b-44bb-a8d2-0507502f069f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:14.398882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ece2d-735b-44bb-a8d2-0507502f069f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:14.410382 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 551/1090] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:14.416529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad297e-6229-4633-9c3f-e6c46aa322d5 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:14.424624 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b42b2-c7a5-4e0b-9188-9fee818d440f None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:14.425077 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b42b2-c7a5-4e0b-9188-9fee818d440f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:14.425226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b42b2-c7a5-4e0b-9188-9fee818d440f None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:14.426546 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 552/1091] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:19:14.431551 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79af32-ab2b-40a6-96a2-fa40b149c7b5 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:14.431832 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79af32-ab2b-40a6-96a2-fa40b149c7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:14.432112 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79af32-ab2b-40a6-96a2-fa40b149c7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:14.450365 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1d79af32-ab2b-40a6-96a2-fa40b149c7b5 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:14.451556 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1d79af32-ab2b-40a6-96a2-fa40b149c7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:14.517355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 553/1092] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:19:14.523760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad297e-6229-4633-9c3f-e6c46aa322d5 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:14.551705 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:14.556913 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e5c83-342f-4481-81f3-6c4255abbf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:14.557576 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e5c83-342f-4481-81f3-6c4255abbf47 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:14.561896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e5c83-342f-4481-81f3-6c4255abbf47 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:14.561896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e5c83-342f-4481-81f3-6c4255abbf47 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:14.565289 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e5c83-342f-4481-81f3-6c4255abbf47 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:14.566720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e5c83-342f-4481-81f3-6c4255abbf47 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:14.578070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-fb3e5c83-342f-4481-81f3-6c4255abbf47 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 10:19:14.578070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:14.578070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:14.578070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:14.578070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:14.578070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:14.578070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:14.578070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:14.578070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:14.578070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:14.578070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:14.578070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:14.578070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:19:14.578070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:19:14.578070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:19:14.578070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:19:14.579038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:14.579038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:14.579038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:19:14.579038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:19:14.579038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:19:14.579038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:19:14.579038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:14.579038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:14.579038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:14.579038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:14.579038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:14.579038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:14.579038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:14.579038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:14.579038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:14.579038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:14.579038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:14.579038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:14.585493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:14.585493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:14.585493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:19:14.585493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:19:14.585493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:19:14.585493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:19:14.585493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:19:14.585493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:19:14.585493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 10:19:14.585493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:14.585493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 540/1093] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Sep 19 10:19:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:14.586863 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79af32-ab2b-40a6-96a2-fa40b149c7b5 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:14.630316 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:14.682872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79af32-ab2b-40a6-96a2-fa40b149c7b5 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:14.715654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692ee10-510b-4ef6-98bf-0000524ef2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:14.716252 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692ee10-510b-4ef6-98bf-0000524ef2a7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:14.716530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692ee10-510b-4ef6-98bf-0000524ef2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:14.716803 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692ee10-510b-4ef6-98bf-0000524ef2a7 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:14.717558 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d692ee10-510b-4ef6-98bf-0000524ef2a7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:14.717870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d692ee10-510b-4ef6-98bf-0000524ef2a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:14.719274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d692ee10-510b-4ef6-98bf-0000524ef2a7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:14.736232 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 554/1094] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Sep 19 10:19:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:14.741889 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e5c83-342f-4481-81f3-6c4255abbf47 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:14.790487 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692ee10-510b-4ef6-98bf-0000524ef2a7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:14.843676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e5c83-342f-4481-81f3-6c4255abbf47 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:14.876344 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca2a02-e500-4b3d-a24f-98a25bb3e003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:14.876944 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca2a02-e500-4b3d-a24f-98a25bb3e003 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:14.877241 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca2a02-e500-4b3d-a24f-98a25bb3e003 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:14.877519 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca2a02-e500-4b3d-a24f-98a25bb3e003 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:14.892936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692ee10-510b-4ef6-98bf-0000524ef2a7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:14.894493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:14.927145 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dc9bc-1f94-4663-b26a-ed1a420c5f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:14.927771 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dc9bc-1f94-4663-b26a-ed1a420c5f83 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:14.928571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dc9bc-1f94-4663-b26a-ed1a420c5f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:14.928978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dc9bc-1f94-4663-b26a-ed1a420c5f83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:14.935981 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:14.939425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01dc9bc-1f94-4663-b26a-ed1a420c5f83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:19:14.944853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-c01dc9bc-1f94-4663-b26a-ed1a420c5f83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:19:14.944853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:14.944853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:14.944853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:14.944853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:14.944853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:14.944853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:14.944853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:14.944853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:14.944853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:14.944853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:14.944853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:14.944853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:19:14.944853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:19:14.944853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:19:14.944853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:19:14.946820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:19:14.946820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:19:14.946820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:19:14.946820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:19:14.946820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:14.946820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:14.946820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:14.946820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:14.946820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:14.946820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:14.946820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:14.946820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:14.946820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:14.946820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:14.946820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:14.946820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:14.948133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:14.948133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:14.948133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:14.948133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:14.948133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:19:14.948133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:19:14.948133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:19:14.948133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:19:14.948133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:19:14.948133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:19:14.948133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:19:14.948133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:19:14.948947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca2a02-e500-4b3d-a24f-98a25bb3e003 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:14.949049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 555/1095] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:19:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:19:14.949172 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca2a02-e500-4b3d-a24f-98a25bb3e003 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:14.951840 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dc9bc-1f94-4663-b26a-ed1a420c5f83 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:15.047970 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 541/1096] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:19:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:19:15.053703 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca2a02-e500-4b3d-a24f-98a25bb3e003 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:15.055715 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dc9bc-1f94-4663-b26a-ed1a420c5f83 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:15.089428 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573824c-c796-4efc-8b30-ccaa244d2c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:15.090023 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573824c-c796-4efc-8b30-ccaa244d2c0d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:15.090309 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573824c-c796-4efc-8b30-ccaa244d2c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:15.090582 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573824c-c796-4efc-8b30-ccaa244d2c0d None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:15.091237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573824c-c796-4efc-8b30-ccaa244d2c0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:15.091546 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573824c-c796-4efc-8b30-ccaa244d2c0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:15.092900 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573824c-c796-4efc-8b30-ccaa244d2c0d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:15.106186 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 556/1097] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:19:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:15.107108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573824c-c796-4efc-8b30-ccaa244d2c0d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:15.133242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:15.142371 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:15.152102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca2a02-e500-4b3d-a24f-98a25bb3e003 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:15.186981 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebdd54-ec5f-4462-88af-3e55db763040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:15.187601 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebdd54-ec5f-4462-88af-3e55db763040 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:15.187879 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebdd54-ec5f-4462-88af-3e55db763040 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:15.188151 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebdd54-ec5f-4462-88af-3e55db763040 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:15.198377 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ebdd54-ec5f-4462-88af-3e55db763040 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:15.199818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ebdd54-ec5f-4462-88af-3e55db763040 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:15.206885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573824c-c796-4efc-8b30-ccaa244d2c0d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:15.214769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-d3ebdd54-ec5f-4462-88af-3e55db763040 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:19:15.214769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:15.214769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:15.214769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:15.214769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:15.214769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:15.214769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:15.214769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:15.214769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:15.214769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:15.214769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:15.214769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:15.214769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:19:15.214769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:19:15.214769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:19:15.214769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:19:15.215815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:15.215815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:15.215815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:15.215815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:15.215815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:15.215815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:15.215815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:15.215815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:15.215815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:15.215815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:15.215815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:15.215815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:15.215815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:15.215815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:15.215815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:15.215815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:15.215815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:19:15.216722 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:19:15.216722 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:19:15.216722 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:15.216722 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 542/1098] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:19:15.221621 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebdd54-ec5f-4462-88af-3e55db763040 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:15.242491 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed5202-0093-407c-91ba-2e3bb1fc9e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:15.243116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed5202-0093-407c-91ba-2e3bb1fc9e00 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:15.243393 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed5202-0093-407c-91ba-2e3bb1fc9e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:15.243669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed5202-0093-407c-91ba-2e3bb1fc9e00 None admin] PATH_INFO: `/v3/users/da16d6d541664d9e9f250f2aa3aeb21b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:15.244814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed5202-0093-407c-91ba-2e3bb1fc9e00 None admin] RBAC: Authorizing `identity:get_user(user_id=da16d6d541664d9e9f250f2aa3aeb21b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:15.246257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed5202-0093-407c-91ba-2e3bb1fc9e00 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:15.247863 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 557/1099] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:19:15 2022] GET /identity/v3/users/da16d6d541664d9e9f250f2aa3aeb21b => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:15.253364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed5202-0093-407c-91ba-2e3bb1fc9e00 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:15.319972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebdd54-ec5f-4462-88af-3e55db763040 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:15.341975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:15.351300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed5202-0093-407c-91ba-2e3bb1fc9e00 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:15.353994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6325d7-a9e9-4dc8-b71e-d3fc4c87d5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:15.354594 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6325d7-a9e9-4dc8-b71e-d3fc4c87d5d0 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:15.354868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6325d7-a9e9-4dc8-b71e-d3fc4c87d5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:15.355218 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6325d7-a9e9-4dc8-b71e-d3fc4c87d5d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:15.355847 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6325d7-a9e9-4dc8-b71e-d3fc4c87d5d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:15.356144 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6325d7-a9e9-4dc8-b71e-d3fc4c87d5d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:15.357581 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6325d7-a9e9-4dc8-b71e-d3fc4c87d5d0 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:15.369635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 543/1100] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:15.375327 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6325d7-a9e9-4dc8-b71e-d3fc4c87d5d0 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:15.377955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:15.386189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f7ab9-46f6-4855-9d20-dce17566f982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:15.392662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f7ab9-46f6-4855-9d20-dce17566f982 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:15.392662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f7ab9-46f6-4855-9d20-dce17566f982 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:15.392662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f7ab9-46f6-4855-9d20-dce17566f982 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:15.473847 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6325d7-a9e9-4dc8-b71e-d3fc4c87d5d0 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:15.506962 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f7ab9-46f6-4855-9d20-dce17566f982 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:15.508419 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f7ab9-46f6-4855-9d20-dce17566f982 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:15.508638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05fe74-c221-43cd-b45e-699d8a65a6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:15.509304 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05fe74-c221-43cd-b45e-699d8a65a6d7 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:15.509582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05fe74-c221-43cd-b45e-699d8a65a6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:15.509854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05fe74-c221-43cd-b45e-699d8a65a6d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:15.516618 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05fe74-c221-43cd-b45e-699d8a65a6d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:15.518078 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05fe74-c221-43cd-b45e-699d8a65a6d7 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:15.531419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-8b05fe74-c221-43cd-b45e-699d8a65a6d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:19:15.531419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:15.531419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:15.531419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:15.531419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:15.531419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:15.531419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:15.531419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:15.531419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:15.531419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:15.531419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:15.531419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:15.531419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:19:15.531419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:19:15.531419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:19:15.532504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:19:15.532504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:15.532504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:15.532504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:15.532504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:15.532504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:15.532504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:15.532504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:15.532504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:15.532504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:15.532504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:15.532504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:15.532504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:15.532504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:15.532504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:15.532504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:15.532504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:15.533771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:19:15.533771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:19:15.533771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:19:15.533771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:19:15.533771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:19:15.533771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:19:15.533771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:19:15.533771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:15.533771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 544/1101] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:19:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:19:15.543299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05fe74-c221-43cd-b45e-699d8a65a6d7 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:15.608580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:15.609274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:15.625169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 558/1102] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:19:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 10:19:15.626409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f7ab9-46f6-4855-9d20-dce17566f982 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:15.648310 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05fe74-c221-43cd-b45e-699d8a65a6d7 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:15.680584 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6538f4b-bb56-41ee-a4d8-8b9efbc63a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:15.681307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6538f4b-bb56-41ee-a4d8-8b9efbc63a90 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:15.681581 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6538f4b-bb56-41ee-a4d8-8b9efbc63a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:15.681973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6538f4b-bb56-41ee-a4d8-8b9efbc63a90 None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:15.682654 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6538f4b-bb56-41ee-a4d8-8b9efbc63a90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:15.682954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6538f4b-bb56-41ee-a4d8-8b9efbc63a90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:15.684381 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6538f4b-bb56-41ee-a4d8-8b9efbc63a90 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:15.696329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 545/1103] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:19:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:15.702285 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6538f4b-bb56-41ee-a4d8-8b9efbc63a90 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:15.720693 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f7ab9-46f6-4855-9d20-dce17566f982 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:15.754042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c31221-1214-43be-a057-d339fa5c8352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:15.754658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c31221-1214-43be-a057-d339fa5c8352 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:15.754933 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c31221-1214-43be-a057-d339fa5c8352 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:15.755205 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c31221-1214-43be-a057-d339fa5c8352 None admin] PATH_INFO: `/v3/projects/45a08b39cc00409597972684860476e0` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:15.756824 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c31221-1214-43be-a057-d339fa5c8352 None admin] RBAC: Authorizing `identity:get_project(project_id=45a08b39cc00409597972684860476e0)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:15.758267 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c31221-1214-43be-a057-d339fa5c8352 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:15.759870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 559/1104] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:19:15 2022] GET /identity/v3/projects/45a08b39cc00409597972684860476e0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:15.765754 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c31221-1214-43be-a057-d339fa5c8352 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:15.802533 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6538f4b-bb56-41ee-a4d8-8b9efbc63a90 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:15.822993 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:15.827408 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:15.840954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aced5c6-e226-4a7d-a5bb-30aa3d5aa187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:15.841598 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aced5c6-e226-4a7d-a5bb-30aa3d5aa187 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:15.841879 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aced5c6-e226-4a7d-a5bb-30aa3d5aa187 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:15.842161 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aced5c6-e226-4a7d-a5bb-30aa3d5aa187 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:15.842888 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aced5c6-e226-4a7d-a5bb-30aa3d5aa187 None admin] RBAC: Adding query filter params (role.id=ccde793bf8ca438a97cf89d53e0a0f1e, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=b4007794a30247c4a318af62d05e3b92) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:15.843196 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aced5c6-e226-4a7d-a5bb-30aa3d5aa187 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:15.844644 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aced5c6-e226-4a7d-a5bb-30aa3d5aa187 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:15.850978 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 546/1105] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:19:15 2022] GET /identity/v3/role_assignments?user.id=b4007794a30247c4a318af62d05e3b92&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=ccde793bf8ca438a97cf89d53e0a0f1e => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:15.871264 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c31221-1214-43be-a057-d339fa5c8352 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:15.910952 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0a005-5b5e-465a-a041-f813740ad977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:15.911580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0a005-5b5e-465a-a041-f813740ad977 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:15.911863 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0a005-5b5e-465a-a041-f813740ad977 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:15.912140 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0a005-5b5e-465a-a041-f813740ad977 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:15.912931 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0a005-5b5e-465a-a041-f813740ad977 None admin] RBAC: Adding query filter params (role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.project.id=45a08b39cc00409597972684860476e0, user.id=da16d6d541664d9e9f250f2aa3aeb21b) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:15.913277 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0a005-5b5e-465a-a041-f813740ad977 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:15.914747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0a005-5b5e-465a-a041-f813740ad977 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:15.921340 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 560/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:19:15 2022] GET /identity/v3/role_assignments?user.id=da16d6d541664d9e9f250f2aa3aeb21b&scope.project.id=45a08b39cc00409597972684860476e0&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:18.150688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e6e2c-088e-4682-aaf1-076f7bc6e8e8 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:18.151213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e6e2c-088e-4682-aaf1-076f7bc6e8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:18.151213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e6e2c-088e-4682-aaf1-076f7bc6e8e8 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:18.152571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 547/1107] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:18.157877 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14733db0-b104-4cc8-a36d-51ea2259e6c8 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:18.158271 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14733db0-b104-4cc8-a36d-51ea2259e6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:18.158429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14733db0-b104-4cc8-a36d-51ea2259e6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:18.176117 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-14733db0-b104-4cc8-a36d-51ea2259e6c8 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:18.177370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-14733db0-b104-4cc8-a36d-51ea2259e6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:18.241926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 561/1108] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:18.512335 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e6e2c-088e-4682-aaf1-076f7bc6e8e8 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:18.611380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e6e2c-088e-4682-aaf1-076f7bc6e8e8 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:18.643557 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50ea20-1808-4ae4-9052-c7fcec90e558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:18.644157 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50ea20-1808-4ae4-9052-c7fcec90e558 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:18.644429 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50ea20-1808-4ae4-9052-c7fcec90e558 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:18.644691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50ea20-1808-4ae4-9052-c7fcec90e558 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:18.645794 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a50ea20-1808-4ae4-9052-c7fcec90e558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:18.648057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a50ea20-1808-4ae4-9052-c7fcec90e558 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:18.649586 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 548/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:19:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:18.655331 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14733db0-b104-4cc8-a36d-51ea2259e6c8 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:18.708069 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:18.751500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14733db0-b104-4cc8-a36d-51ea2259e6c8 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:18.784008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643a9e9-93c3-41f9-ae98-6e8664e2d072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:18.784544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643a9e9-93c3-41f9-ae98-6e8664e2d072 None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:18.784827 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643a9e9-93c3-41f9-ae98-6e8664e2d072 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:18.785128 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643a9e9-93c3-41f9-ae98-6e8664e2d072 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:18.785776 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643a9e9-93c3-41f9-ae98-6e8664e2d072 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:18.787785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643a9e9-93c3-41f9-ae98-6e8664e2d072 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:18.870686 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 562/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 19 10:19:18 2022] POST /identity/v3/users => generated 286 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:19:21.036089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519fffa-b197-42a8-a734-8fb47664c006 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:21.036628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519fffa-b197-42a8-a734-8fb47664c006 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:21.036628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519fffa-b197-42a8-a734-8fb47664c006 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:21.038822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 549/1111] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:21.043940 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a6567-558a-4b50-87fb-e4b5ebffba74 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:21.044353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a6567-558a-4b50-87fb-e4b5ebffba74 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:21.044490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a6567-558a-4b50-87fb-e4b5ebffba74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:21.062148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-362a6567-558a-4b50-87fb-e4b5ebffba74 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:21.063330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-362a6567-558a-4b50-87fb-e4b5ebffba74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:21.127565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 563/1112] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:21.399732 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519fffa-b197-42a8-a734-8fb47664c006 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:21.453996 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:21.497412 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519fffa-b197-42a8-a734-8fb47664c006 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:21.530766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27332101-e332-49b2-84b0-517cdbbf4b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:21.531381 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27332101-e332-49b2-84b0-517cdbbf4b1e None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:21.531657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27332101-e332-49b2-84b0-517cdbbf4b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:21.531926 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27332101-e332-49b2-84b0-517cdbbf4b1e None admin] PATH_INFO: `/v3/roles/e4309dcb11f74e10ba108b7b00264476` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:21.533312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27332101-e332-49b2-84b0-517cdbbf4b1e None admin] RBAC: Authorizing `identity:get_role(role_id=e4309dcb11f74e10ba108b7b00264476)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:21.534667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27332101-e332-49b2-84b0-517cdbbf4b1e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:21.535964 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 550/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:19:21 2022] GET /identity/v3/roles/e4309dcb11f74e10ba108b7b00264476 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:21.541682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a6567-558a-4b50-87fb-e4b5ebffba74 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:21.582590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:21.637926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a6567-558a-4b50-87fb-e4b5ebffba74 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:21.669917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5474395-59af-4524-a1c5-e373efc695e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:21.670531 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5474395-59af-4524-a1c5-e373efc695e1 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:21.670807 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5474395-59af-4524-a1c5-e373efc695e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:21.671080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5474395-59af-4524-a1c5-e373efc695e1 None admin] PATH_INFO: `/v3/users/05177141620f4822b85c0cdebb6fa4a1` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:21.684102 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5474395-59af-4524-a1c5-e373efc695e1 None admin] RBAC: Authorizing `identity:get_user(user_id=05177141620f4822b85c0cdebb6fa4a1)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:21.685552 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5474395-59af-4524-a1c5-e373efc695e1 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:21.687130 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 564/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:19:21 2022] GET /identity/v3/users/05177141620f4822b85c0cdebb6fa4a1 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:21.692864 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27332101-e332-49b2-84b0-517cdbbf4b1e None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:21.788153 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27332101-e332-49b2-84b0-517cdbbf4b1e None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:21.810167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:21.822283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9b747-c94b-4d38-b5b7-a7d096ff49c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:21.822895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9b747-c94b-4d38-b5b7-a7d096ff49c5 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:21.823178 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9b747-c94b-4d38-b5b7-a7d096ff49c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:21.823481 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9b747-c94b-4d38-b5b7-a7d096ff49c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:21.889129 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9b747-c94b-4d38-b5b7-a7d096ff49c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:21.890569 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9b747-c94b-4d38-b5b7-a7d096ff49c5 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:21.987955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 551/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:19:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:19:21.994403 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5474395-59af-4524-a1c5-e373efc695e1 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:22.085001 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:22.088625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5474395-59af-4524-a1c5-e373efc695e1 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:22.122082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b56d8-dc3e-4106-a01f-176a0526bbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:22.122691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b56d8-dc3e-4106-a01f-176a0526bbbd None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:22.122970 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b56d8-dc3e-4106-a01f-176a0526bbbd None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:22.123243 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b56d8-dc3e-4106-a01f-176a0526bbbd None admin] PATH_INFO: `/v3/projects/45a08b39cc00409597972684860476e0` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:22.124401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b56d8-dc3e-4106-a01f-176a0526bbbd None admin] RBAC: Authorizing `identity:get_project(project_id=45a08b39cc00409597972684860476e0)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:22.125840 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b56d8-dc3e-4106-a01f-176a0526bbbd None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:22.127451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 565/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:19:21 2022] GET /identity/v3/projects/45a08b39cc00409597972684860476e0 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:22.133615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9b747-c94b-4d38-b5b7-a7d096ff49c5 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:22.156642 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:22.230775 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9b747-c94b-4d38-b5b7-a7d096ff49c5 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:22.263846 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e271afc-9913-432f-baec-fc8380599a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:22.264380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e271afc-9913-432f-baec-fc8380599a81 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:22.264656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e271afc-9913-432f-baec-fc8380599a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:22.264926 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e271afc-9913-432f-baec-fc8380599a81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:22.265667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e271afc-9913-432f-baec-fc8380599a81 None admin] RBAC: Adding query filter params (role.id=e4309dcb11f74e10ba108b7b00264476, scope.project.id=45a08b39cc00409597972684860476e0, user.id=05177141620f4822b85c0cdebb6fa4a1) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:22.265979 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e271afc-9913-432f-baec-fc8380599a81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:22.267387 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e271afc-9913-432f-baec-fc8380599a81 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:22.273482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 552/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:19:22 2022] GET /identity/v3/role_assignments?user.id=05177141620f4822b85c0cdebb6fa4a1&scope.project.id=45a08b39cc00409597972684860476e0&role.id=e4309dcb11f74e10ba108b7b00264476 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:24.378128 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdf9e2-d85c-47ed-b855-55ce92f24460 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:24.378658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdf9e2-d85c-47ed-b855-55ce92f24460 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:24.378658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdf9e2-d85c-47ed-b855-55ce92f24460 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:24.380011 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 566/1118] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:24.385703 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59806df-d8c3-441a-850c-1b0d316b95c0 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:24.386143 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59806df-d8c3-441a-850c-1b0d316b95c0 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:24.386236 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59806df-d8c3-441a-850c-1b0d316b95c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:24.404712 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-d59806df-d8c3-441a-850c-1b0d316b95c0 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:24.405938 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d59806df-d8c3-441a-850c-1b0d316b95c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:24.469841 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 553/1119] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:24.739061 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fdf9e2-d85c-47ed-b855-55ce92f24460 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:24.837466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fdf9e2-d85c-47ed-b855-55ce92f24460 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:24.869830 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0918bd-ce8f-4b8b-9130-a5506f97221a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:24.870439 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0918bd-ce8f-4b8b-9130-a5506f97221a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:24.870714 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0918bd-ce8f-4b8b-9130-a5506f97221a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:24.870987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0918bd-ce8f-4b8b-9130-a5506f97221a None admin] PATH_INFO: `/v3/roles/e4309dcb11f74e10ba108b7b00264476` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:24.872328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0918bd-ce8f-4b8b-9130-a5506f97221a None admin] RBAC: Authorizing `identity:get_role(role_id=e4309dcb11f74e10ba108b7b00264476)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:24.873704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0918bd-ce8f-4b8b-9130-a5506f97221a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:24.875028 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 567/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:19:24 2022] GET /identity/v3/roles/e4309dcb11f74e10ba108b7b00264476 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:24.881451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59806df-d8c3-441a-850c-1b0d316b95c0 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:24.898329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:24.977682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59806df-d8c3-441a-850c-1b0d316b95c0 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:25.010838 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0be9e9-a360-402d-980a-fc05720b3030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:25.011384 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0be9e9-a360-402d-980a-fc05720b3030 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:25.011658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0be9e9-a360-402d-980a-fc05720b3030 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:25.011925 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0be9e9-a360-402d-980a-fc05720b3030 None admin] PATH_INFO: `/v3/users/05177141620f4822b85c0cdebb6fa4a1` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:25.013016 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0be9e9-a360-402d-980a-fc05720b3030 None admin] RBAC: Authorizing `identity:get_user(user_id=05177141620f4822b85c0cdebb6fa4a1)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:25.014451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0be9e9-a360-402d-980a-fc05720b3030 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:25.016001 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 554/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:19:24 2022] GET /identity/v3/users/05177141620f4822b85c0cdebb6fa4a1 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:25.022062 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0918bd-ce8f-4b8b-9130-a5506f97221a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:25.055847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:25.118366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0918bd-ce8f-4b8b-9130-a5506f97221a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:25.151630 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094c98b-6676-4ac9-b2e0-01fc1c11e7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:25.152238 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094c98b-6676-4ac9-b2e0-01fc1c11e7fb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:25.152520 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094c98b-6676-4ac9-b2e0-01fc1c11e7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:25.152797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094c98b-6676-4ac9-b2e0-01fc1c11e7fb None admin] PATH_INFO: `/v3/projects/45a08b39cc00409597972684860476e0` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:25.153934 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094c98b-6676-4ac9-b2e0-01fc1c11e7fb None admin] RBAC: Authorizing `identity:get_project(project_id=45a08b39cc00409597972684860476e0)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:25.155352 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094c98b-6676-4ac9-b2e0-01fc1c11e7fb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:25.156950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 568/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:19:25 2022] GET /identity/v3/projects/45a08b39cc00409597972684860476e0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:25.162809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0be9e9-a360-402d-980a-fc05720b3030 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:25.242156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:25.257439 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0be9e9-a360-402d-980a-fc05720b3030 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:25.290038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f475d-f5a9-4d59-ae55-464ac5b8151e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:25.290619 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f475d-f5a9-4d59-ae55-464ac5b8151e None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:25.290894 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f475d-f5a9-4d59-ae55-464ac5b8151e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:25.291163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f475d-f5a9-4d59-ae55-464ac5b8151e None admin] PATH_INFO: `/v3/projects/45a08b39cc00409597972684860476e0/users/05177141620f4822b85c0cdebb6fa4a1/roles/e4309dcb11f74e10ba108b7b00264476` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:25.293276 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f475d-f5a9-4d59-ae55-464ac5b8151e None admin] RBAC: Authorizing `identity:create_grant(project_id=45a08b39cc00409597972684860476e0, user_id=05177141620f4822b85c0cdebb6fa4a1, role_id=e4309dcb11f74e10ba108b7b00264476)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:25.294686 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f475d-f5a9-4d59-ae55-464ac5b8151e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:25.355373 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 555/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:19:25 2022] PUT /identity/v3/projects/45a08b39cc00409597972684860476e0/users/05177141620f4822b85c0cdebb6fa4a1/roles/e4309dcb11f74e10ba108b7b00264476 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:19:27.478636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed06ba-835d-429b-98f0-b41a4460fa5e None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:27.479188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed06ba-835d-429b-98f0-b41a4460fa5e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:27.479188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed06ba-835d-429b-98f0-b41a4460fa5e None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:27.480533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 569/1124] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:27.485691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9a951-d1c5-4b96-b6a1-a229efca723b None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:27.486072 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9a951-d1c5-4b96-b6a1-a229efca723b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:27.486247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9a951-d1c5-4b96-b6a1-a229efca723b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:27.508318 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-19a9a951-d1c5-4b96-b6a1-a229efca723b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:27.509552 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-19a9a951-d1c5-4b96-b6a1-a229efca723b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:27.574481 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 556/1125] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:27.843227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed06ba-835d-429b-98f0-b41a4460fa5e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:27.941591 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed06ba-835d-429b-98f0-b41a4460fa5e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:27.943676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:27.977607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4650e-cfe7-45d5-9f44-7f20ea4724df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:27.978222 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4650e-cfe7-45d5-9f44-7f20ea4724df None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:27.978503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4650e-cfe7-45d5-9f44-7f20ea4724df None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:27.978780 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4650e-cfe7-45d5-9f44-7f20ea4724df None admin] PATH_INFO: `/v3/roles/e4309dcb11f74e10ba108b7b00264476` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:27.980160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4650e-cfe7-45d5-9f44-7f20ea4724df None admin] RBAC: Authorizing `identity:get_role(role_id=e4309dcb11f74e10ba108b7b00264476)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:27.981563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4650e-cfe7-45d5-9f44-7f20ea4724df None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:27.983133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 570/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:19:27 2022] GET /identity/v3/roles/e4309dcb11f74e10ba108b7b00264476 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:27.988652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9a951-d1c5-4b96-b6a1-a229efca723b None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:28.059505 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:28.083977 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9a951-d1c5-4b96-b6a1-a229efca723b None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:28.117847 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17768c3b-a2df-4c3e-aef1-97222eec63c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:28.118468 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17768c3b-a2df-4c3e-aef1-97222eec63c3 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:28.118748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17768c3b-a2df-4c3e-aef1-97222eec63c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:28.119022 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17768c3b-a2df-4c3e-aef1-97222eec63c3 None admin] PATH_INFO: `/v3/users/05177141620f4822b85c0cdebb6fa4a1` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:28.120131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17768c3b-a2df-4c3e-aef1-97222eec63c3 None admin] RBAC: Authorizing `identity:get_user(user_id=05177141620f4822b85c0cdebb6fa4a1)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:28.121589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17768c3b-a2df-4c3e-aef1-97222eec63c3 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:28.123158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 557/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:19:27 2022] GET /identity/v3/users/05177141620f4822b85c0cdebb6fa4a1 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:28.128895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4650e-cfe7-45d5-9f44-7f20ea4724df None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:28.224718 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4650e-cfe7-45d5-9f44-7f20ea4724df None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:28.257068 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a0471-6fae-4bb4-8bc1-cf10e14256b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:28.257683 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a0471-6fae-4bb4-8bc1-cf10e14256b7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:28.257962 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a0471-6fae-4bb4-8bc1-cf10e14256b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:28.258234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a0471-6fae-4bb4-8bc1-cf10e14256b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:28.295598 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:28.323766 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0a0471-6fae-4bb4-8bc1-cf10e14256b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:28.325260 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0a0471-6fae-4bb4-8bc1-cf10e14256b7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:28.417453 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 571/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:19:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:19:28.423352 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17768c3b-a2df-4c3e-aef1-97222eec63c3 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:28.519508 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17768c3b-a2df-4c3e-aef1-97222eec63c3 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:28.552448 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95b15a-d9ae-4424-ba4d-29e5f9012246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:28.553089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95b15a-d9ae-4424-ba4d-29e5f9012246 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:28.553364 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95b15a-d9ae-4424-ba4d-29e5f9012246 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:28.553634 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95b15a-d9ae-4424-ba4d-29e5f9012246 None admin] PATH_INFO: `/v3/projects/45a08b39cc00409597972684860476e0` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:28.554744 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95b15a-d9ae-4424-ba4d-29e5f9012246 None admin] RBAC: Authorizing `identity:get_project(project_id=45a08b39cc00409597972684860476e0)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:28.556173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95b15a-d9ae-4424-ba4d-29e5f9012246 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:28.557791 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 558/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:19:28 2022] GET /identity/v3/projects/45a08b39cc00409597972684860476e0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:28.563912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a0471-6fae-4bb4-8bc1-cf10e14256b7 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:28.629184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:28.659385 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a0471-6fae-4bb4-8bc1-cf10e14256b7 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:28.691921 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76288578-dbb5-4389-ba7c-a82bb6206f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:28.692453 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76288578-dbb5-4389-ba7c-a82bb6206f5c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:28.693310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76288578-dbb5-4389-ba7c-a82bb6206f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:28.693588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76288578-dbb5-4389-ba7c-a82bb6206f5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:28.694303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76288578-dbb5-4389-ba7c-a82bb6206f5c None admin] RBAC: Adding query filter params (role.id=e4309dcb11f74e10ba108b7b00264476, scope.project.id=45a08b39cc00409597972684860476e0, user.id=05177141620f4822b85c0cdebb6fa4a1) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:28.694615 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76288578-dbb5-4389-ba7c-a82bb6206f5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:28.696023 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76288578-dbb5-4389-ba7c-a82bb6206f5c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:28.701588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 572/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:19:28 2022] GET /identity/v3/role_assignments?user.id=05177141620f4822b85c0cdebb6fa4a1&scope.project.id=45a08b39cc00409597972684860476e0&role.id=e4309dcb11f74e10ba108b7b00264476 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:30.870246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e606d59-0881-42d0-83be-f3010d960031 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:30.870836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e606d59-0881-42d0-83be-f3010d960031 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:30.870836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e606d59-0881-42d0-83be-f3010d960031 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:30.872410 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 559/1131] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:30.877800 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6c5c7-30aa-4dc2-8b99-660919df0eea None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:30.878218 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6c5c7-30aa-4dc2-8b99-660919df0eea None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:30.878356 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6c5c7-30aa-4dc2-8b99-660919df0eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:30.897216 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-45c6c5c7-30aa-4dc2-8b99-660919df0eea None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:30.898419 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-45c6c5c7-30aa-4dc2-8b99-660919df0eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:30.963328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 573/1132] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:31.232667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e606d59-0881-42d0-83be-f3010d960031 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:31.244844 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:31.333770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e606d59-0881-42d0-83be-f3010d960031 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:31.366532 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af457e-abe1-4741-8fac-912047968a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:31.367148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af457e-abe1-4741-8fac-912047968a94 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:31.367425 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af457e-abe1-4741-8fac-912047968a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:31.367692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af457e-abe1-4741-8fac-912047968a94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:31.368813 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af457e-abe1-4741-8fac-912047968a94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:31.370262 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af457e-abe1-4741-8fac-912047968a94 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:31.371803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 560/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:19:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:31.377613 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6c5c7-30aa-4dc2-8b99-660919df0eea None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:31.436383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:31.472463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6c5c7-30aa-4dc2-8b99-660919df0eea None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:31.504606 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb54fa-f75b-4ea5-b718-7c80ffde46ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:31.505760 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb54fa-f75b-4ea5-b718-7c80ffde46ba None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:31.506160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb54fa-f75b-4ea5-b718-7c80ffde46ba None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:31.506518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb54fa-f75b-4ea5-b718-7c80ffde46ba None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:31.507159 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb54fa-f75b-4ea5-b718-7c80ffde46ba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:31.508568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb54fa-f75b-4ea5-b718-7c80ffde46ba None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:31.648425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 574/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 19 10:19:31 2022] POST /identity/v3/projects => generated 311 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:19:33.820774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81a1af-9fbe-43fd-ab94-a0679d0e9944 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:33.821337 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81a1af-9fbe-43fd-ab94-a0679d0e9944 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:33.821337 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81a1af-9fbe-43fd-ab94-a0679d0e9944 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:33.822682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 561/1135] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:33.828118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900f256-3a52-4b0e-9183-b147c79a93d6 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:33.828607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900f256-3a52-4b0e-9183-b147c79a93d6 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:33.828709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900f256-3a52-4b0e-9183-b147c79a93d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:33.846462 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c900f256-3a52-4b0e-9183-b147c79a93d6 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:33.847649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c900f256-3a52-4b0e-9183-b147c79a93d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:33.912991 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 575/1136] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:34.186938 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81a1af-9fbe-43fd-ab94-a0679d0e9944 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:34.264323 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:34.286582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81a1af-9fbe-43fd-ab94-a0679d0e9944 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:34.320801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93c53f-7b9d-41fa-9846-b5fc1caf9591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:34.321448 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93c53f-7b9d-41fa-9846-b5fc1caf9591 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:34.321728 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93c53f-7b9d-41fa-9846-b5fc1caf9591 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:34.322000 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93c53f-7b9d-41fa-9846-b5fc1caf9591 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:34.323100 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93c53f-7b9d-41fa-9846-b5fc1caf9591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:34.324525 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93c53f-7b9d-41fa-9846-b5fc1caf9591 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:34.326113 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 562/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:19:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:34.332122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900f256-3a52-4b0e-9183-b147c79a93d6 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:34.376674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:34.431806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900f256-3a52-4b0e-9183-b147c79a93d6 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:34.467091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca73481-fbc1-448b-a2d3-236947ac429a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:34.467746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca73481-fbc1-448b-a2d3-236947ac429a None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:34.468027 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca73481-fbc1-448b-a2d3-236947ac429a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:34.468304 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca73481-fbc1-448b-a2d3-236947ac429a None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:34.468950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca73481-fbc1-448b-a2d3-236947ac429a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:34.470411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca73481-fbc1-448b-a2d3-236947ac429a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:34.578108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 576/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 19 10:19:34 2022] POST /identity/v3/users => generated 286 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:19:36.736633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114ab6f-6471-4af7-9b1a-58650752ef83 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:36.737178 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114ab6f-6471-4af7-9b1a-58650752ef83 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:36.737178 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114ab6f-6471-4af7-9b1a-58650752ef83 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:36.738516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 563/1139] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:36.744191 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0b9f3-7151-4352-b723-6395707267ef None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:36.744584 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0b9f3-7151-4352-b723-6395707267ef None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:36.744741 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0b9f3-7151-4352-b723-6395707267ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:36.763054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4be0b9f3-7151-4352-b723-6395707267ef None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:36.764230 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4be0b9f3-7151-4352-b723-6395707267ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:36.828704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 577/1140] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:37.095895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114ab6f-6471-4af7-9b1a-58650752ef83 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:37.195036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114ab6f-6471-4af7-9b1a-58650752ef83 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:37.227530 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f3d27-26f5-4968-9312-b0b528aded9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:37.228144 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f3d27-26f5-4968-9312-b0b528aded9a None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:37.228422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f3d27-26f5-4968-9312-b0b528aded9a None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:37.228699 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f3d27-26f5-4968-9312-b0b528aded9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:37.238739 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f3d27-26f5-4968-9312-b0b528aded9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:19:37.243566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-8f9f3d27-26f5-4968-9312-b0b528aded9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:19:37.243566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:37.243566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:37.243566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:37.243566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:37.243566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:37.243566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:37.243566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:37.243566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:37.243566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:37.243566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:37.243566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:37.243566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:19:37.243566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:19:37.243566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:19:37.243566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:19:37.244760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:19:37.244760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:19:37.244760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:19:37.244760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:19:37.244760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:37.244760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:37.244760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:37.244760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:37.244760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:37.244760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:37.244760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:37.244760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:37.244760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:37.244760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:37.244760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:37.244760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:37.245929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:37.245929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:37.245929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:37.245929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:37.245929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:19:37.245929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:19:37.245929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:19:37.245929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:19:37.245929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:19:37.245929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:19:37.245929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:19:37.245929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:37.245929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 564/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:19:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:37.250862 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0b9f3-7151-4352-b723-6395707267ef None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:37.280925 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:37.346106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0b9f3-7151-4352-b723-6395707267ef None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:37.378166 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49691ce4-162f-4074-853b-d30cffc65852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:37.378755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49691ce4-162f-4074-853b-d30cffc65852 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:37.379031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49691ce4-162f-4074-853b-d30cffc65852 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:37.379302 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49691ce4-162f-4074-853b-d30cffc65852 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:37.379954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49691ce4-162f-4074-853b-d30cffc65852 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:37.380252 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49691ce4-162f-4074-853b-d30cffc65852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:37.382207 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49691ce4-162f-4074-853b-d30cffc65852 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:37.389871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 578/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:19:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:37.395963 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f3d27-26f5-4968-9312-b0b528aded9a None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:37.405076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:37.492182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f3d27-26f5-4968-9312-b0b528aded9a None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:37.524984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5aa91-4356-46d8-baf9-2a0d767a3fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:37.525620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5aa91-4356-46d8-baf9-2a0d767a3fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:37.525892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5aa91-4356-46d8-baf9-2a0d767a3fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:37.526159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5aa91-4356-46d8-baf9-2a0d767a3fc6 None admin] PATH_INFO: `/v3/users/459087d6b2a745418fb4a1eea339915e` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:37.538635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5aa91-4356-46d8-baf9-2a0d767a3fc6 None admin] RBAC: Authorizing `identity:get_user(user_id=459087d6b2a745418fb4a1eea339915e)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:37.540050 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5aa91-4356-46d8-baf9-2a0d767a3fc6 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:37.541650 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 565/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:19:37 2022] GET /identity/v3/users/459087d6b2a745418fb4a1eea339915e => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:37.547371 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49691ce4-162f-4074-853b-d30cffc65852 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:37.636513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:37.642753 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49691ce4-162f-4074-853b-d30cffc65852 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:37.678596 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebd1e9-c6b9-448d-a65d-001662ebc046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:37.679198 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebd1e9-c6b9-448d-a65d-001662ebc046 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:37.679475 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebd1e9-c6b9-448d-a65d-001662ebc046 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:37.679746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebd1e9-c6b9-448d-a65d-001662ebc046 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:37.746279 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ebd1e9-c6b9-448d-a65d-001662ebc046 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:37.747721 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ebd1e9-c6b9-448d-a65d-001662ebc046 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:37.840405 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:37.847834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 579/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:19:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:19:37.853882 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5aa91-4356-46d8-baf9-2a0d767a3fc6 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:37.914251 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:37.949998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5aa91-4356-46d8-baf9-2a0d767a3fc6 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:37.986174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403a3b3-38a9-4c12-8313-1782fae17d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:37.986793 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403a3b3-38a9-4c12-8313-1782fae17d1c None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:37.987070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403a3b3-38a9-4c12-8313-1782fae17d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:37.987341 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403a3b3-38a9-4c12-8313-1782fae17d1c None admin] PATH_INFO: `/v3/projects/0a68f07d55a54b86a2045c8d17b0ae3d` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:37.988454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403a3b3-38a9-4c12-8313-1782fae17d1c None admin] RBAC: Authorizing `identity:get_project(project_id=0a68f07d55a54b86a2045c8d17b0ae3d)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:37.989894 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403a3b3-38a9-4c12-8313-1782fae17d1c None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:37.991532 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 566/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:19:37 2022] GET /identity/v3/projects/0a68f07d55a54b86a2045c8d17b0ae3d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:37.997810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebd1e9-c6b9-448d-a65d-001662ebc046 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:38.093328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebd1e9-c6b9-448d-a65d-001662ebc046 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:38.125904 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809099ff-cc00-485f-a926-b7764b6df7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:38.126492 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809099ff-cc00-485f-a926-b7764b6df7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:38.126773 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809099ff-cc00-485f-a926-b7764b6df7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:38.127044 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809099ff-cc00-485f-a926-b7764b6df7d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:38.127757 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809099ff-cc00-485f-a926-b7764b6df7d0 None admin] RBAC: Adding query filter params (role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.project.id=0a68f07d55a54b86a2045c8d17b0ae3d, user.id=459087d6b2a745418fb4a1eea339915e) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:38.128068 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809099ff-cc00-485f-a926-b7764b6df7d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:38.129518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809099ff-cc00-485f-a926-b7764b6df7d0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:38.135250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 580/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:19:37 2022] GET /identity/v3/role_assignments?user.id=459087d6b2a745418fb4a1eea339915e&scope.project.id=0a68f07d55a54b86a2045c8d17b0ae3d&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:40.240982 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4bddd-3e2d-455e-aa90-1e0bb3b47612 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:40.241564 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4bddd-3e2d-455e-aa90-1e0bb3b47612 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:40.241564 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4bddd-3e2d-455e-aa90-1e0bb3b47612 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:40.242868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 567/1147] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:40.248203 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3ce36-1586-429f-a0f9-e6968942e084 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:40.248620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3ce36-1586-429f-a0f9-e6968942e084 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:40.248800 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3ce36-1586-429f-a0f9-e6968942e084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:40.266884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5ac3ce36-1586-429f-a0f9-e6968942e084 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:40.268061 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5ac3ce36-1586-429f-a0f9-e6968942e084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:40.320045 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:40.333041 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 581/1148] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:40.603931 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4bddd-3e2d-455e-aa90-1e0bb3b47612 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:40.707511 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4bddd-3e2d-455e-aa90-1e0bb3b47612 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:40.736695 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:40.741428 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc744bb3-bf5c-41b0-b2e3-ed6438180544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:40.742045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc744bb3-bf5c-41b0-b2e3-ed6438180544 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:40.742323 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc744bb3-bf5c-41b0-b2e3-ed6438180544 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:40.742598 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc744bb3-bf5c-41b0-b2e3-ed6438180544 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:40.752307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc744bb3-bf5c-41b0-b2e3-ed6438180544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:19:40.757674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-fc744bb3-bf5c-41b0-b2e3-ed6438180544 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:19:40.757674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:40.757674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:40.757674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:40.757674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:40.757674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:40.757674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:40.757674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:40.757674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:40.757674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:40.757674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:40.757674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:40.757674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:19:40.757674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:19:40.757674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:19:40.757674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:19:40.758801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:19:40.758801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:19:40.758801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:19:40.758801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:19:40.758801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:40.758801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:40.758801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:40.758801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:40.758801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:40.758801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:40.758801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:40.758801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:40.758801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:40.758801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:40.758801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:40.758801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:40.759809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:40.759809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:40.759809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:40.759809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:40.759809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:19:40.759809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:19:40.759809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:19:40.759809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:19:40.759809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:19:40.759809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:19:40.759809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:19:40.759809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:40.759809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 568/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:19:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:40.764787 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3ce36-1586-429f-a0f9-e6968942e084 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:40.859865 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3ce36-1586-429f-a0f9-e6968942e084 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:40.892012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46d63d-b4ce-4156-aab2-6d9cbe70f7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:40.892604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46d63d-b4ce-4156-aab2-6d9cbe70f7cb None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:40.892883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46d63d-b4ce-4156-aab2-6d9cbe70f7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:40.893189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46d63d-b4ce-4156-aab2-6d9cbe70f7cb None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:40.893842 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46d63d-b4ce-4156-aab2-6d9cbe70f7cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:40.894144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46d63d-b4ce-4156-aab2-6d9cbe70f7cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:40.895781 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46d63d-b4ce-4156-aab2-6d9cbe70f7cb None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:40.905310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 582/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:19:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:40.911174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc744bb3-bf5c-41b0-b2e3-ed6438180544 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:41.008228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc744bb3-bf5c-41b0-b2e3-ed6438180544 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:41.041637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6936d3-4171-48b3-badb-5592b39215b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:41.042250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6936d3-4171-48b3-badb-5592b39215b6 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:41.042527 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6936d3-4171-48b3-badb-5592b39215b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:41.042801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6936d3-4171-48b3-badb-5592b39215b6 None admin] PATH_INFO: `/v3/users/459087d6b2a745418fb4a1eea339915e` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:41.043902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6936d3-4171-48b3-badb-5592b39215b6 None admin] RBAC: Authorizing `identity:get_user(user_id=459087d6b2a745418fb4a1eea339915e)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:41.045354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6936d3-4171-48b3-badb-5592b39215b6 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:41.046898 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 569/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:19:40 2022] GET /identity/v3/users/459087d6b2a745418fb4a1eea339915e => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:41.053179 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46d63d-b4ce-4156-aab2-6d9cbe70f7cb None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:41.100149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:41.147566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46d63d-b4ce-4156-aab2-6d9cbe70f7cb None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:41.180350 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67b813-ee7c-4f86-833c-a7b903c1bff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:41.180964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67b813-ee7c-4f86-833c-a7b903c1bff2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:41.181274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67b813-ee7c-4f86-833c-a7b903c1bff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:41.181553 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67b813-ee7c-4f86-833c-a7b903c1bff2 None admin] PATH_INFO: `/v3/projects/0a68f07d55a54b86a2045c8d17b0ae3d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:41.182661 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67b813-ee7c-4f86-833c-a7b903c1bff2 None admin] RBAC: Authorizing `identity:get_project(project_id=0a68f07d55a54b86a2045c8d17b0ae3d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:41.184078 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67b813-ee7c-4f86-833c-a7b903c1bff2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:41.185714 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 583/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:19:41 2022] GET /identity/v3/projects/0a68f07d55a54b86a2045c8d17b0ae3d => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:41.191471 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6936d3-4171-48b3-badb-5592b39215b6 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:41.248845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:41.290879 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6936d3-4171-48b3-badb-5592b39215b6 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:41.323823 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780d5a2-bcc1-42f3-bbc4-86ac7699fddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:41.324466 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780d5a2-bcc1-42f3-bbc4-86ac7699fddd None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:41.324741 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780d5a2-bcc1-42f3-bbc4-86ac7699fddd None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:41.325008 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780d5a2-bcc1-42f3-bbc4-86ac7699fddd None admin] PATH_INFO: `/v3/projects/0a68f07d55a54b86a2045c8d17b0ae3d/users/459087d6b2a745418fb4a1eea339915e/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:41.327929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780d5a2-bcc1-42f3-bbc4-86ac7699fddd None admin] RBAC: Authorizing `identity:create_grant(project_id=0a68f07d55a54b86a2045c8d17b0ae3d, user_id=459087d6b2a745418fb4a1eea339915e, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:41.329435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780d5a2-bcc1-42f3-bbc4-86ac7699fddd None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:41.394581 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 570/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:19:41 2022] PUT /identity/v3/projects/0a68f07d55a54b86a2045c8d17b0ae3d/users/459087d6b2a745418fb4a1eea339915e/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:19:43.491058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799ecea-15e5-4a8b-a91a-e304c55421e0 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:43.491616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799ecea-15e5-4a8b-a91a-e304c55421e0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:43.491616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799ecea-15e5-4a8b-a91a-e304c55421e0 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:43.492938 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 584/1154] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:43.498667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469973e5-af5a-4b1e-9a97-325c5f0a11d6 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:43.499059 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469973e5-af5a-4b1e-9a97-325c5f0a11d6 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:43.499203 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469973e5-af5a-4b1e-9a97-325c5f0a11d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:43.518271 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-469973e5-af5a-4b1e-9a97-325c5f0a11d6 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:43.519471 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-469973e5-af5a-4b1e-9a97-325c5f0a11d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:43.583083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 571/1155] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 10:19:43.850286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799ecea-15e5-4a8b-a91a-e304c55421e0 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:43.948444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799ecea-15e5-4a8b-a91a-e304c55421e0 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:43.965399 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:43.981149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62178299-c6b0-4fd4-b4ff-bb3365f1d58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:43.983954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62178299-c6b0-4fd4-b4ff-bb3365f1d58d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:43.983954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62178299-c6b0-4fd4-b4ff-bb3365f1d58d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:43.983954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62178299-c6b0-4fd4-b4ff-bb3365f1d58d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:43.992527 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62178299-c6b0-4fd4-b4ff-bb3365f1d58d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:19:43.997695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-62178299-c6b0-4fd4-b4ff-bb3365f1d58d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:19:43.997695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:43.997695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:43.997695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:43.997695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:43.997695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:43.997695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:43.997695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:43.997695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:43.997695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:43.997695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:43.997695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:43.997695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:19:43.997695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:19:43.997695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:19:43.997695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:19:43.998829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:19:43.998829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:19:43.998829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:19:43.998829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:19:43.998829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:43.998829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:43.998829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:43.998829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:43.998829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:43.998829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:43.998829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:43.998829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:43.998829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:43.998829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:43.998829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:43.998829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:43.999915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:43.999915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:43.999915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:43.999915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:43.999915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:19:43.999915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:19:43.999915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:19:43.999915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:19:43.999915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:19:43.999915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:19:43.999915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:19:43.999915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:19:43.999915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 585/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:19:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:44.005037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469973e5-af5a-4b1e-9a97-325c5f0a11d6 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:44.054367 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:44.101689 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469973e5-af5a-4b1e-9a97-325c5f0a11d6 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:44.135392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950acc3-4d08-4910-b133-cba868cb7ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:44.135989 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950acc3-4d08-4910-b133-cba868cb7ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:44.136270 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950acc3-4d08-4910-b133-cba868cb7ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:44.136546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950acc3-4d08-4910-b133-cba868cb7ea9 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:44.137237 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950acc3-4d08-4910-b133-cba868cb7ea9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:44.137542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950acc3-4d08-4910-b133-cba868cb7ea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:44.138910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950acc3-4d08-4910-b133-cba868cb7ea9 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:44.146900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 572/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:19:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:44.152878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62178299-c6b0-4fd4-b4ff-bb3365f1d58d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:44.247094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62178299-c6b0-4fd4-b4ff-bb3365f1d58d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:44.279718 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a020c-4ff6-4cd4-a314-a6298286f288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:44.280337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a020c-4ff6-4cd4-a314-a6298286f288 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:44.280617 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a020c-4ff6-4cd4-a314-a6298286f288 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:44.280891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a020c-4ff6-4cd4-a314-a6298286f288 None admin] PATH_INFO: `/v3/users/459087d6b2a745418fb4a1eea339915e` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:44.282032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a020c-4ff6-4cd4-a314-a6298286f288 None admin] RBAC: Authorizing `identity:get_user(user_id=459087d6b2a745418fb4a1eea339915e)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:44.283454 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a020c-4ff6-4cd4-a314-a6298286f288 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:44.285009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 586/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:19:44 2022] GET /identity/v3/users/459087d6b2a745418fb4a1eea339915e => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:44.290736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950acc3-4d08-4910-b133-cba868cb7ea9 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:44.387493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950acc3-4d08-4910-b133-cba868cb7ea9 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:44.399866 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:44.420208 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc01d94-9170-4a2d-a607-995f9b7b9ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:44.420799 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc01d94-9170-4a2d-a607-995f9b7b9ace None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:44.421103 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc01d94-9170-4a2d-a607-995f9b7b9ace None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:44.421382 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc01d94-9170-4a2d-a607-995f9b7b9ace None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:44.486241 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc01d94-9170-4a2d-a607-995f9b7b9ace None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:44.487676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc01d94-9170-4a2d-a607-995f9b7b9ace None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:44.587453 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 573/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:19:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:19:44.593421 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a020c-4ff6-4cd4-a314-a6298286f288 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:44.639158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:44.691332 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a020c-4ff6-4cd4-a314-a6298286f288 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:44.723945 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff125f1-e42d-41df-8df1-90d46cb3e4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:44.724558 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff125f1-e42d-41df-8df1-90d46cb3e4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:44.724840 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff125f1-e42d-41df-8df1-90d46cb3e4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:44.725141 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff125f1-e42d-41df-8df1-90d46cb3e4d7 None admin] PATH_INFO: `/v3/projects/0a68f07d55a54b86a2045c8d17b0ae3d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:44.726248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff125f1-e42d-41df-8df1-90d46cb3e4d7 None admin] RBAC: Authorizing `identity:get_project(project_id=0a68f07d55a54b86a2045c8d17b0ae3d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:44.727657 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff125f1-e42d-41df-8df1-90d46cb3e4d7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:44.729299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 587/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:19:44 2022] GET /identity/v3/projects/0a68f07d55a54b86a2045c8d17b0ae3d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:44.735307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc01d94-9170-4a2d-a607-995f9b7b9ace None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:44.748915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:44.833993 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc01d94-9170-4a2d-a607-995f9b7b9ace None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:44.867071 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e7da5-8dda-4d1e-86f7-09da096c7117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:44.867664 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e7da5-8dda-4d1e-86f7-09da096c7117 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:44.867944 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e7da5-8dda-4d1e-86f7-09da096c7117 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:44.868217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e7da5-8dda-4d1e-86f7-09da096c7117 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:44.868940 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e7da5-8dda-4d1e-86f7-09da096c7117 None admin] RBAC: Adding query filter params (role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.project.id=0a68f07d55a54b86a2045c8d17b0ae3d, user.id=459087d6b2a745418fb4a1eea339915e) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:44.869273 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e7da5-8dda-4d1e-86f7-09da096c7117 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:44.870701 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e7da5-8dda-4d1e-86f7-09da096c7117 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:44.876260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 574/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:19:44 2022] GET /identity/v3/role_assignments?user.id=459087d6b2a745418fb4a1eea339915e&scope.project.id=0a68f07d55a54b86a2045c8d17b0ae3d&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:19:47.033105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430563f-34b3-4ebf-ae22-1d72bed6f956 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:47.033690 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430563f-34b3-4ebf-ae22-1d72bed6f956 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:47.033690 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430563f-34b3-4ebf-ae22-1d72bed6f956 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:47.034966 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 588/1162] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:47.040126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967753f-540d-4ccf-b1b4-1647b211dc54 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:47.040564 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967753f-540d-4ccf-b1b4-1647b211dc54 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:47.040672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967753f-540d-4ccf-b1b4-1647b211dc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:47.058868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-5967753f-540d-4ccf-b1b4-1647b211dc54 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:47.060047 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5967753f-540d-4ccf-b1b4-1647b211dc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:47.123800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 575/1163] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:47.392009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430563f-34b3-4ebf-ae22-1d72bed6f956 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:47.489943 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430563f-34b3-4ebf-ae22-1d72bed6f956 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:47.503032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:47.522024 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c355b8-df2e-42da-b342-0f7c3df6a811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:47.526223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c355b8-df2e-42da-b342-0f7c3df6a811 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:47.526223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c355b8-df2e-42da-b342-0f7c3df6a811 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:47.526223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c355b8-df2e-42da-b342-0f7c3df6a811 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:47.586930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c355b8-df2e-42da-b342-0f7c3df6a811 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:47.588348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c355b8-df2e-42da-b342-0f7c3df6a811 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:47.682073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 589/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:19:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:19:47.687915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967753f-540d-4ccf-b1b4-1647b211dc54 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:47.696359 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:47.784715 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967753f-540d-4ccf-b1b4-1647b211dc54 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:47.818374 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603799e5-e274-4ffe-8dea-dea0a79bd577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:47.819013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603799e5-e274-4ffe-8dea-dea0a79bd577 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:47.819295 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603799e5-e274-4ffe-8dea-dea0a79bd577 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:47.819569 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603799e5-e274-4ffe-8dea-dea0a79bd577 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:47.826274 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603799e5-e274-4ffe-8dea-dea0a79bd577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:47.827713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603799e5-e274-4ffe-8dea-dea0a79bd577 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:47.838878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-603799e5-e274-4ffe-8dea-dea0a79bd577 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 19 10:19:47.838878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:47.838878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 19 10:19:47.838878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 19 10:19:47.838878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:19:47.838878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:19:47.838878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:19:47.838878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:19:47.838878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 19 10:19:47.838878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:47.838878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:19:47.838878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:47.838878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:47.838878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:47.838878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:47.838878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:47.838878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:47.840158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:47.840158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:47.840158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:47.840158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:47.840158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:47.840158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:47.840158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:19:47.840158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:19:47.840158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:19:47.840158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:19:47.840158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:47.840158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:47.840158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:47.840158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:47.840158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:47.840158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:47.841357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:47.841357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:47.841357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:47.841357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:47.841357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:47.841357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:47.841357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:47.841357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:47.841357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:47.841357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:47.841357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 19 10:19:47.841357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:19:47.841357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 19 10:19:47.841357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:19:47.841357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 576/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 19 10:19:47 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:47.845910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c355b8-df2e-42da-b342-0f7c3df6a811 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:47.860388 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:47.939953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c355b8-df2e-42da-b342-0f7c3df6a811 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:47.972034 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c73a6-78d1-4bee-9d80-a801319f62cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:47.972621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c73a6-78d1-4bee-9d80-a801319f62cd None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:47.972896 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c73a6-78d1-4bee-9d80-a801319f62cd None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:47.973193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c73a6-78d1-4bee-9d80-a801319f62cd None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:47.973830 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c73a6-78d1-4bee-9d80-a801319f62cd None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:47.974132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c73a6-78d1-4bee-9d80-a801319f62cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:47.975484 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c73a6-78d1-4bee-9d80-a801319f62cd None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:47.982272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 590/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Sep 19 10:19:47 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:50.096868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced0dd3-46f7-4188-b2b2-2bc4e15ba57d None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:50.097423 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced0dd3-46f7-4188-b2b2-2bc4e15ba57d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:50.097423 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced0dd3-46f7-4188-b2b2-2bc4e15ba57d None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:50.099289 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 577/1167] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:50.104580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454bae8-18b4-43db-b86a-1c07bcad056a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:50.104977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454bae8-18b4-43db-b86a-1c07bcad056a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:50.105166 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454bae8-18b4-43db-b86a-1c07bcad056a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:50.122782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a454bae8-18b4-43db-b86a-1c07bcad056a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:50.123969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a454bae8-18b4-43db-b86a-1c07bcad056a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:50.187876 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 591/1168] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:50.455148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced0dd3-46f7-4188-b2b2-2bc4e15ba57d None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:50.518901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:50.553118 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced0dd3-46f7-4188-b2b2-2bc4e15ba57d None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:50.585889 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989752e-7dc6-47ad-a1ee-0a551442f1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:50.586475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989752e-7dc6-47ad-a1ee-0a551442f1a2 None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:50.586748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989752e-7dc6-47ad-a1ee-0a551442f1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:50.587016 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989752e-7dc6-47ad-a1ee-0a551442f1a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:50.587675 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989752e-7dc6-47ad-a1ee-0a551442f1a2 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:50.589020 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989752e-7dc6-47ad-a1ee-0a551442f1a2 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:50.652777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 578/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Mon Sep 19 10:19:50 2022] POST /identity/v3/domains => generated 267 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:19:52.796007 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e95e4a-1939-4a40-af69-fbb59bc20d46 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:52.796568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e95e4a-1939-4a40-af69-fbb59bc20d46 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:52.796568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e95e4a-1939-4a40-af69-fbb59bc20d46 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:52.797954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 592/1170] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:52.803219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b6378-0fc2-4e2d-8964-5632e1e616de None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:52.803636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b6378-0fc2-4e2d-8964-5632e1e616de None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:52.803742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b6378-0fc2-4e2d-8964-5632e1e616de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:52.823816 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-5c3b6378-0fc2-4e2d-8964-5632e1e616de None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:52.824455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5c3b6378-0fc2-4e2d-8964-5632e1e616de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:52.888766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 579/1171] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:53.160386 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e95e4a-1939-4a40-af69-fbb59bc20d46 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:53.171576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:53.255994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e95e4a-1939-4a40-af69-fbb59bc20d46 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:53.288536 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9035d8-1a27-4249-9d4b-52be3346edf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:53.289166 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9035d8-1a27-4249-9d4b-52be3346edf7 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:53.289442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9035d8-1a27-4249-9d4b-52be3346edf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:53.289712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9035d8-1a27-4249-9d4b-52be3346edf7 None admin] PATH_INFO: `/v3/domains/367f72ee1a914c2e9d547392c6e388bf` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:53.299653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9035d8-1a27-4249-9d4b-52be3346edf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=367f72ee1a914c2e9d547392c6e388bf)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:53.301081 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9035d8-1a27-4249-9d4b-52be3346edf7 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:53.302695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 593/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Mon Sep 19 10:19:53 2022] GET /identity/v3/domains/367f72ee1a914c2e9d547392c6e388bf => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:53.308591 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b6378-0fc2-4e2d-8964-5632e1e616de None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:53.361941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:53.403740 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b6378-0fc2-4e2d-8964-5632e1e616de None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:53.438182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f355b-0932-4c02-996c-d0e44bb472e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:53.438761 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f355b-0932-4c02-996c-d0e44bb472e0 None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:53.439037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f355b-0932-4c02-996c-d0e44bb472e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:53.439306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f355b-0932-4c02-996c-d0e44bb472e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:53.439959 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f355b-0932-4c02-996c-d0e44bb472e0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:53.441440 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f355b-0932-4c02-996c-d0e44bb472e0 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:53.548759 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 580/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 19 10:19:53 2022] POST /identity/v3/projects => generated 361 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:19:55.723150 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1636fd-5b53-45f9-a97f-cc809a6a6a77 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:55.723705 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1636fd-5b53-45f9-a97f-cc809a6a6a77 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:55.723705 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1636fd-5b53-45f9-a97f-cc809a6a6a77 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:55.725036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 594/1174] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:55.730576 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea32a85-2e0b-4dcd-9867-a0a41abcabcc None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:55.730960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea32a85-2e0b-4dcd-9867-a0a41abcabcc None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:55.731130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea32a85-2e0b-4dcd-9867-a0a41abcabcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:55.748805 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-1ea32a85-2e0b-4dcd-9867-a0a41abcabcc None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:55.750018 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-1ea32a85-2e0b-4dcd-9867-a0a41abcabcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:55.813860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 581/1175] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:56.086704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1636fd-5b53-45f9-a97f-cc809a6a6a77 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:56.152003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:56.184304 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1636fd-5b53-45f9-a97f-cc809a6a6a77 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:56.216625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ddd5ad-6c88-4e8b-b769-099b1843a299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:56.217262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddd5ad-6c88-4e8b-b769-099b1843a299 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:56.217541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddd5ad-6c88-4e8b-b769-099b1843a299 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:56.217816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddd5ad-6c88-4e8b-b769-099b1843a299 None admin] PATH_INFO: `/v3/domains/367f72ee1a914c2e9d547392c6e388bf` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:56.218910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ddd5ad-6c88-4e8b-b769-099b1843a299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=367f72ee1a914c2e9d547392c6e388bf)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:56.220323 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ddd5ad-6c88-4e8b-b769-099b1843a299 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:56.221874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 595/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Mon Sep 19 10:19:56 2022] GET /identity/v3/domains/367f72ee1a914c2e9d547392c6e388bf => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:56.227481 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea32a85-2e0b-4dcd-9867-a0a41abcabcc None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:56.265268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:56.328066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea32a85-2e0b-4dcd-9867-a0a41abcabcc None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:56.361292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0f8d1-07bf-4935-b48a-a4559289c4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:56.361827 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0f8d1-07bf-4935-b48a-a4559289c4b9 None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:56.362101 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0f8d1-07bf-4935-b48a-a4559289c4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:56.362369 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0f8d1-07bf-4935-b48a-a4559289c4b9 None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:56.363003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e0f8d1-07bf-4935-b48a-a4559289c4b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:56.364495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e0f8d1-07bf-4935-b48a-a4559289c4b9 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:56.450894 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 582/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 19 10:19:56 2022] POST /identity/v3/users => generated 311 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:19:58.639519 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ca503-141a-4533-bcf6-11034118a534 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:58.640147 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ca503-141a-4533-bcf6-11034118a534 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:58.640147 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ca503-141a-4533-bcf6-11034118a534 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:58.641443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 596/1178] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:19:58.647356 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2b824-69d7-4215-a41d-c011b31166c6 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:58.647771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2b824-69d7-4215-a41d-c011b31166c6 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:58.647887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2b824-69d7-4215-a41d-c011b31166c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:58.666158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-07f2b824-69d7-4215-a41d-c011b31166c6 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:19:58.667334 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-07f2b824-69d7-4215-a41d-c011b31166c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:19:58.731364 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 583/1179] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:19:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:19:58.999971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ca503-141a-4533-bcf6-11034118a534 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:59.096933 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ca503-141a-4533-bcf6-11034118a534 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:59.129109 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618edae-1e7e-4f5c-88e5-892025a39c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:59.129714 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618edae-1e7e-4f5c-88e5-892025a39c32 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:59.129987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618edae-1e7e-4f5c-88e5-892025a39c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:59.130256 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618edae-1e7e-4f5c-88e5-892025a39c32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:59.132762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:59.140184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618edae-1e7e-4f5c-88e5-892025a39c32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:19:59.145726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-b618edae-1e7e-4f5c-88e5-892025a39c32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:19:59.145726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:19:59.145726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:19:59.145726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:19:59.145726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:19:59.145726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:19:59.145726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:19:59.145726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:19:59.145726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:19:59.145726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:19:59.145726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:19:59.145726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:19:59.145726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:19:59.145726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:19:59.145726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:19:59.145726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:19:59.157431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:19:59.157431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:19:59.157431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:19:59.157431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:19:59.157431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:19:59.157431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:19:59.157431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:19:59.157431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:19:59.157431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:19:59.157431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:19:59.157431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:19:59.157431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:19:59.157431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:19:59.157431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:19:59.157431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:19:59.157431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:19:59.158497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:19:59.158497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:19:59.158497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:19:59.158497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:19:59.158497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:19:59.158497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:19:59.158497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:19:59.158497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:19:59.158497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:19:59.158497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:19:59.158497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:19:59.158497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:19:59.158497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 597/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:19:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:19:59.159402 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2b824-69d7-4215-a41d-c011b31166c6 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:59.176819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:59.248816 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2b824-69d7-4215-a41d-c011b31166c6 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:59.281812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc658c-9cc5-49f7-b9b0-47a6df5cb238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:59.282354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc658c-9cc5-49f7-b9b0-47a6df5cb238 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:59.282633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc658c-9cc5-49f7-b9b0-47a6df5cb238 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:59.282908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc658c-9cc5-49f7-b9b0-47a6df5cb238 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:59.283567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc658c-9cc5-49f7-b9b0-47a6df5cb238 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:19:59.283872 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc658c-9cc5-49f7-b9b0-47a6df5cb238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:59.285830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc658c-9cc5-49f7-b9b0-47a6df5cb238 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:59.293574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 584/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:19:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:59.299402 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618edae-1e7e-4f5c-88e5-892025a39c32 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:59.395925 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618edae-1e7e-4f5c-88e5-892025a39c32 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:59.428366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cc77e-54ae-4fc0-8cff-24ad48442b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:59.428970 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cc77e-54ae-4fc0-8cff-24ad48442b39 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:59.429270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cc77e-54ae-4fc0-8cff-24ad48442b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:59.429540 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cc77e-54ae-4fc0-8cff-24ad48442b39 None admin] PATH_INFO: `/v3/users/4f3cca3d286b4328879a1a75513a86ef` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:59.442526 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5cc77e-54ae-4fc0-8cff-24ad48442b39 None admin] RBAC: Authorizing `identity:get_user(user_id=4f3cca3d286b4328879a1a75513a86ef)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:59.443938 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5cc77e-54ae-4fc0-8cff-24ad48442b39 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:59.446888 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 598/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:19:59 2022] GET /identity/v3/users/4f3cca3d286b4328879a1a75513a86ef => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:59.451219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc658c-9cc5-49f7-b9b0-47a6df5cb238 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:59.537159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:59.551431 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc658c-9cc5-49f7-b9b0-47a6df5cb238 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:59.585216 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb75b6-1343-4d2e-a81b-85a74b4ac8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:59.585815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb75b6-1343-4d2e-a81b-85a74b4ac8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:59.586095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb75b6-1343-4d2e-a81b-85a74b4ac8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:59.586369 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb75b6-1343-4d2e-a81b-85a74b4ac8d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:59.651282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb75b6-1343-4d2e-a81b-85a74b4ac8d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:59.652722 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb75b6-1343-4d2e-a81b-85a74b4ac8d8 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:59.731443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:59.746122 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 585/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:19:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:19:59.752041 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cc77e-54ae-4fc0-8cff-24ad48442b39 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:59.806108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:19:59.849728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cc77e-54ae-4fc0-8cff-24ad48442b39 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:19:59.882127 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9c202-6243-474b-9daa-0cfc6e74d33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:19:59.882738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9c202-6243-474b-9daa-0cfc6e74d33c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:19:59.883018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9c202-6243-474b-9daa-0cfc6e74d33c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:19:59.883294 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9c202-6243-474b-9daa-0cfc6e74d33c None admin] PATH_INFO: `/v3/projects/6d44b7b6e2cf4969ab738721f39b0606` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:19:59.884393 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9c202-6243-474b-9daa-0cfc6e74d33c None admin] RBAC: Authorizing `identity:get_project(project_id=6d44b7b6e2cf4969ab738721f39b0606)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:19:59.885832 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9c202-6243-474b-9daa-0cfc6e74d33c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:19:59.887537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 599/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:19:59 2022] GET /identity/v3/projects/6d44b7b6e2cf4969ab738721f39b0606 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:19:59.893573 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb75b6-1343-4d2e-a81b-85a74b4ac8d8 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:19:59.990109 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb75b6-1343-4d2e-a81b-85a74b4ac8d8 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:20:00.023159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b3866-1198-44a2-b3c6-a02ab133abc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:20:00.023752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b3866-1198-44a2-b3c6-a02ab133abc5 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:20:00.024033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b3866-1198-44a2-b3c6-a02ab133abc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:20:00.024305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b3866-1198-44a2-b3c6-a02ab133abc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:20:00.025022 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b3866-1198-44a2-b3c6-a02ab133abc5 None admin] RBAC: Adding query filter params (role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.project.id=6d44b7b6e2cf4969ab738721f39b0606, user.id=4f3cca3d286b4328879a1a75513a86ef) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:20:00.025363 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b3866-1198-44a2-b3c6-a02ab133abc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:20:00.026780 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b3866-1198-44a2-b3c6-a02ab133abc5 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:20:00.031927 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 586/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:19:59 2022] GET /identity/v3/role_assignments?user.id=4f3cca3d286b4328879a1a75513a86ef&scope.project.id=6d44b7b6e2cf4969ab738721f39b0606&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:20:02.143870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583475d-a28d-4005-a9ab-85be6fe31634 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:20:02.144686 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583475d-a28d-4005-a9ab-85be6fe31634 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:20:02.144686 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583475d-a28d-4005-a9ab-85be6fe31634 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:20:02.145979 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 600/1186] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:20:02.151134 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633578e-2eec-4813-9575-7f7de891b52a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:20:02.151533 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633578e-2eec-4813-9575-7f7de891b52a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:20:02.151671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633578e-2eec-4813-9575-7f7de891b52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:20:02.169350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-a633578e-2eec-4813-9575-7f7de891b52a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:20:02.170520 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-a633578e-2eec-4813-9575-7f7de891b52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:20:02.215384 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:20:02.235204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 587/1187] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:20:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:20:02.512718 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583475d-a28d-4005-a9ab-85be6fe31634 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:20:02.612336 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583475d-a28d-4005-a9ab-85be6fe31634 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:20:02.634132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:20:02.645925 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9433c-920b-4417-891e-bcc12bad231d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:20:02.646535 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9433c-920b-4417-891e-bcc12bad231d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:20:02.646808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9433c-920b-4417-891e-bcc12bad231d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:20:02.647101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9433c-920b-4417-891e-bcc12bad231d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:20:02.656643 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9433c-920b-4417-891e-bcc12bad231d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:20:02.661861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-7fc9433c-920b-4417-891e-bcc12bad231d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:20:02.661861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:20:02.661861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:20:02.661861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:20:02.661861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:20:02.661861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:20:02.661861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:20:02.661861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:20:02.661861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:20:02.661861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:20:02.661861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:20:02.661861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:20:02.661861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:20:02.661861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:20:02.661861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:20:02.661861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:20:02.662812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:20:02.662812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:20:02.662812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:20:02.662812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:20:02.662812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:20:02.662812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:20:02.662812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:20:02.662812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:20:02.662812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:20:02.662812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:20:02.662812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:20:02.662812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:20:02.662812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:20:02.662812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:20:02.662812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:20:02.662812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:20:02.663704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:20:02.663704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:20:02.663704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:20:02.663704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:20:02.663704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:20:02.663704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:20:02.663704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:20:02.663704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:20:02.663704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:20:02.663704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:20:02.663704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:20:02.663704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:20:02.663704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 601/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:20:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:20:02.669234 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633578e-2eec-4813-9575-7f7de891b52a None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:20:02.764405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633578e-2eec-4813-9575-7f7de891b52a None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:20:02.797390 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c32cec-3179-4dbc-8d1f-0fedc13013f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:20:02.797983 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c32cec-3179-4dbc-8d1f-0fedc13013f5 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:20:02.798263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c32cec-3179-4dbc-8d1f-0fedc13013f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:20:02.798539 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c32cec-3179-4dbc-8d1f-0fedc13013f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:20:02.799197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c32cec-3179-4dbc-8d1f-0fedc13013f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:20:02.799498 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c32cec-3179-4dbc-8d1f-0fedc13013f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:20:02.801408 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c32cec-3179-4dbc-8d1f-0fedc13013f5 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:20:02.809137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 588/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:20:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:20:02.815016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9433c-920b-4417-891e-bcc12bad231d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:20:02.910963 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9433c-920b-4417-891e-bcc12bad231d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:20:02.943774 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e9d74-e462-45b4-b033-61a0a788bdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:20:02.944396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e9d74-e462-45b4-b033-61a0a788bdc9 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:20:02.944657 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e9d74-e462-45b4-b033-61a0a788bdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:20:02.944929 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e9d74-e462-45b4-b033-61a0a788bdc9 None admin] PATH_INFO: `/v3/users/4f3cca3d286b4328879a1a75513a86ef` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:20:02.946057 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e9d74-e462-45b4-b033-61a0a788bdc9 None admin] RBAC: Authorizing `identity:get_user(user_id=4f3cca3d286b4328879a1a75513a86ef)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:20:02.947474 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e9d74-e462-45b4-b033-61a0a788bdc9 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:20:02.949016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 602/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:20:02 2022] GET /identity/v3/users/4f3cca3d286b4328879a1a75513a86ef => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:20:02.954765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c32cec-3179-4dbc-8d1f-0fedc13013f5 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:20:02.996532 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:20:03.052014 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c32cec-3179-4dbc-8d1f-0fedc13013f5 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:20:03.084326 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb5880-c4b3-4944-9062-91c877e1910e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:20:03.084940 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb5880-c4b3-4944-9062-91c877e1910e None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:20:03.085310 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb5880-c4b3-4944-9062-91c877e1910e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:20:03.085580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb5880-c4b3-4944-9062-91c877e1910e None admin] PATH_INFO: `/v3/projects/6d44b7b6e2cf4969ab738721f39b0606` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:20:03.086677 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb5880-c4b3-4944-9062-91c877e1910e None admin] RBAC: Authorizing `identity:get_project(project_id=6d44b7b6e2cf4969ab738721f39b0606)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:20:03.088087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb5880-c4b3-4944-9062-91c877e1910e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:20:03.089716 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 589/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:20:02 2022] GET /identity/v3/projects/6d44b7b6e2cf4969ab738721f39b0606 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:20:03.095573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e9d74-e462-45b4-b033-61a0a788bdc9 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:20:03.145378 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:20:03.193497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e9d74-e462-45b4-b033-61a0a788bdc9 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:20:03.224477 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f70d3f-1811-4d4f-9a0c-4d9c6d605183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:20:03.225132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f70d3f-1811-4d4f-9a0c-4d9c6d605183 None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:20:03.225412 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f70d3f-1811-4d4f-9a0c-4d9c6d605183 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:20:03.225683 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f70d3f-1811-4d4f-9a0c-4d9c6d605183 None admin] PATH_INFO: `/v3/projects/6d44b7b6e2cf4969ab738721f39b0606/users/4f3cca3d286b4328879a1a75513a86ef/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:20:03.227523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f70d3f-1811-4d4f-9a0c-4d9c6d605183 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d44b7b6e2cf4969ab738721f39b0606, user_id=4f3cca3d286b4328879a1a75513a86ef, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:20:03.228914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f70d3f-1811-4d4f-9a0c-4d9c6d605183 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:20:03.289933 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 603/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:20:03 2022] PUT /identity/v3/projects/6d44b7b6e2cf4969ab738721f39b0606/users/4f3cca3d286b4328879a1a75513a86ef/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:20:05.391405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65225241-ba13-4814-a0d8-1df30b4da73b None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:20:05.391956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65225241-ba13-4814-a0d8-1df30b4da73b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:20:05.391956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65225241-ba13-4814-a0d8-1df30b4da73b None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:20:05.393302 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 590/1193] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:20:05.399469 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b20cc-a81b-4df4-b096-ae85c310484b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:20:05.399850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b20cc-a81b-4df4-b096-ae85c310484b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:20:05.400020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b20cc-a81b-4df4-b096-ae85c310484b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:20:05.417748 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e89b20cc-a81b-4df4-b096-ae85c310484b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:20:05.431459 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e89b20cc-a81b-4df4-b096-ae85c310484b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:20:05.483219 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 604/1194] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:20:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:20:05.757063 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65225241-ba13-4814-a0d8-1df30b4da73b None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:20:05.855462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65225241-ba13-4814-a0d8-1df30b4da73b None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:20:05.865920 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:20:05.887862 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7717c0-12e2-4ed5-bb7c-a7a5ff3cfcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:20:05.888483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7717c0-12e2-4ed5-bb7c-a7a5ff3cfcc4 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:20:05.888757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7717c0-12e2-4ed5-bb7c-a7a5ff3cfcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:20:05.889029 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7717c0-12e2-4ed5-bb7c-a7a5ff3cfcc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:20:05.899740 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7717c0-12e2-4ed5-bb7c-a7a5ff3cfcc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:20:05.904544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-0e7717c0-12e2-4ed5-bb7c-a7a5ff3cfcc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:20:05.904544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:20:05.904544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:20:05.904544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:20:05.904544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:20:05.904544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:20:05.904544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:20:05.904544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:20:05.904544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:20:05.904544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:20:05.904544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:20:05.904544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:20:05.904544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:20:05.904544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:20:05.904544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:20:05.904544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:20:05.905671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:20:05.905671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:20:05.905671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:20:05.905671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:20:05.905671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:20:05.905671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:20:05.905671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:20:05.905671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:20:05.905671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:20:05.905671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:20:05.905671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:20:05.905671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:20:05.905671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:20:05.905671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:20:05.905671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:20:05.905671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:20:05.906526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:20:05.906526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:20:05.906526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:20:05.906526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:20:05.906526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:20:05.906526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:20:05.906526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:20:05.906526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:20:05.906526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:20:05.906526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:20:05.906526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:20:05.906526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:20:05.906526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 591/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:20:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:20:05.912215 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b20cc-a81b-4df4-b096-ae85c310484b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:20:05.951965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:20:06.007462 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b20cc-a81b-4df4-b096-ae85c310484b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:20:06.040174 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e21e75-ec65-41e9-8546-3ff56a7fe311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:20:06.040766 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e21e75-ec65-41e9-8546-3ff56a7fe311 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:20:06.041045 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e21e75-ec65-41e9-8546-3ff56a7fe311 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:20:06.041349 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e21e75-ec65-41e9-8546-3ff56a7fe311 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:20:06.042003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e21e75-ec65-41e9-8546-3ff56a7fe311 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:20:06.042307 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e21e75-ec65-41e9-8546-3ff56a7fe311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:20:06.043647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e21e75-ec65-41e9-8546-3ff56a7fe311 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:20:06.051982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 605/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:20:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:20:06.057876 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7717c0-12e2-4ed5-bb7c-a7a5ff3cfcc4 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:20:06.155586 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7717c0-12e2-4ed5-bb7c-a7a5ff3cfcc4 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:20:06.188028 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1a12e-9e4b-4b5c-af54-e4ffc2196e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:20:06.188636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1a12e-9e4b-4b5c-af54-e4ffc2196e9b None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:20:06.188912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1a12e-9e4b-4b5c-af54-e4ffc2196e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:20:06.189215 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1a12e-9e4b-4b5c-af54-e4ffc2196e9b None admin] PATH_INFO: `/v3/users/4f3cca3d286b4328879a1a75513a86ef` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:20:06.190307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1a12e-9e4b-4b5c-af54-e4ffc2196e9b None admin] RBAC: Authorizing `identity:get_user(user_id=4f3cca3d286b4328879a1a75513a86ef)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:20:06.192358 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1a12e-9e4b-4b5c-af54-e4ffc2196e9b None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:20:06.193919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 592/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Sep 19 10:20:06 2022] GET /identity/v3/users/4f3cca3d286b4328879a1a75513a86ef => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:20:06.199955 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e21e75-ec65-41e9-8546-3ff56a7fe311 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:20:06.293981 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e21e75-ec65-41e9-8546-3ff56a7fe311 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:20:06.295414 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:20:06.326699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805cff99-7052-4678-89b8-1d7886dc48f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:20:06.327284 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cff99-7052-4678-89b8-1d7886dc48f4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:20:06.327562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cff99-7052-4678-89b8-1d7886dc48f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:20:06.327840 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cff99-7052-4678-89b8-1d7886dc48f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:20:06.391625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805cff99-7052-4678-89b8-1d7886dc48f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:20:06.393041 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805cff99-7052-4678-89b8-1d7886dc48f4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:20:06.484904 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 606/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:20:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:20:06.491341 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1a12e-9e4b-4b5c-af54-e4ffc2196e9b None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:20:06.529699 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:20:06.587213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1a12e-9e4b-4b5c-af54-e4ffc2196e9b None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:20:06.620535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0e596-c7c5-4f69-99e9-1bef736bd360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:20:06.621178 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0e596-c7c5-4f69-99e9-1bef736bd360 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:20:06.621458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0e596-c7c5-4f69-99e9-1bef736bd360 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:20:06.621732 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0e596-c7c5-4f69-99e9-1bef736bd360 None admin] PATH_INFO: `/v3/projects/6d44b7b6e2cf4969ab738721f39b0606` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:20:06.622854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf0e596-c7c5-4f69-99e9-1bef736bd360 None admin] RBAC: Authorizing `identity:get_project(project_id=6d44b7b6e2cf4969ab738721f39b0606)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:20:06.624271 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf0e596-c7c5-4f69-99e9-1bef736bd360 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:20:06.625893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 593/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Sep 19 10:20:06 2022] GET /identity/v3/projects/6d44b7b6e2cf4969ab738721f39b0606 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:20:06.632017 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805cff99-7052-4678-89b8-1d7886dc48f4 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:20:06.634219 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:20:06.728699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805cff99-7052-4678-89b8-1d7886dc48f4 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:20:06.760596 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bf32f-f0c5-4613-b76d-93d3917ab5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:20:06.761220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bf32f-f0c5-4613-b76d-93d3917ab5dc None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:20:06.761499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bf32f-f0c5-4613-b76d-93d3917ab5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:20:06.761769 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bf32f-f0c5-4613-b76d-93d3917ab5dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:20:06.762480 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bf32f-f0c5-4613-b76d-93d3917ab5dc None admin] RBAC: Adding query filter params (role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.project.id=6d44b7b6e2cf4969ab738721f39b0606, user.id=4f3cca3d286b4328879a1a75513a86ef) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:20:06.762787 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bf32f-f0c5-4613-b76d-93d3917ab5dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:20:06.764173 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bf32f-f0c5-4613-b76d-93d3917ab5dc None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:20:06.770589 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 607/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:20:06 2022] GET /identity/v3/role_assignments?user.id=4f3cca3d286b4328879a1a75513a86ef&scope.project.id=6d44b7b6e2cf4969ab738721f39b0606&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:20:16.616423 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=47 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:20:16.767244 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=67 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:16.868735 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf4190-b43d-46a3-9430-580536c7f855 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:16.869411 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf4190-b43d-46a3-9430-580536c7f855 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:16.869411 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf4190-b43d-46a3-9430-580536c7f855 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:16.870657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 594/1201] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:21:16.876413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29dbb4-c5bb-4a6f-8613-35c9203be35e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:16.876816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29dbb4-c5bb-4a6f-8613-35c9203be35e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:16.876964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29dbb4-c5bb-4a6f-8613-35c9203be35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:16.895712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-af29dbb4-c5bb-4a6f-8613-35c9203be35e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:21:16.899220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-af29dbb4-c5bb-4a6f-8613-35c9203be35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:21:16.967594 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 608/1202] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:21:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:21:17.239544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf4190-b43d-46a3-9430-580536c7f855 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:17.344863 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf4190-b43d-46a3-9430-580536c7f855 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:17.378421 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb7179-33ef-4f35-804d-68906c2e8f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:17.379043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb7179-33ef-4f35-804d-68906c2e8f24 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:17.379320 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb7179-33ef-4f35-804d-68906c2e8f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:17.379593 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb7179-33ef-4f35-804d-68906c2e8f24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:17.389899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb7179-33ef-4f35-804d-68906c2e8f24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:17.391589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb7179-33ef-4f35-804d-68906c2e8f24 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:17.405881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-aeeb7179-33ef-4f35-804d-68906c2e8f24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:17.405881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:17.405881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:17.405881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:17.405881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:17.405881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:17.405881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:17.405881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:17.405881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:17.405881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:17.405881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:17.405881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:17.405881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:21:17.405881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:21:17.405881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:21:17.405881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:21:17.406956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:17.406956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:17.406956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:17.406956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:17.406956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:17.406956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:17.406956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:17.406956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:17.406956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:17.406956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:17.406956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:17.406956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:17.406956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:17.406956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:17.406956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:17.406956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:17.406956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:21:17.407967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:21:17.407967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:17.407967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:21:17.407967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 595/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:17.413342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29dbb4-c5bb-4a6f-8613-35c9203be35e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:17.509555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29dbb4-c5bb-4a6f-8613-35c9203be35e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:17.537721 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:17.543106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f44777-41ec-4b1c-85d4-a678fa7dabba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:17.543711 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f44777-41ec-4b1c-85d4-a678fa7dabba None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:17.543980 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f44777-41ec-4b1c-85d4-a678fa7dabba None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:17.544257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f44777-41ec-4b1c-85d4-a678fa7dabba None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:17.544896 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f44777-41ec-4b1c-85d4-a678fa7dabba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:17.545234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f44777-41ec-4b1c-85d4-a678fa7dabba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:17.546598 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f44777-41ec-4b1c-85d4-a678fa7dabba None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:17.557383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 609/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:17.563682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb7179-33ef-4f35-804d-68906c2e8f24 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:17.615949 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:17.663701 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb7179-33ef-4f35-804d-68906c2e8f24 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:17.696868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb83d86-d143-48ac-ad48-864d35861a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:17.697496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb83d86-d143-48ac-ad48-864d35861a50 None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:17.697775 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb83d86-d143-48ac-ad48-864d35861a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:17.698045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb83d86-d143-48ac-ad48-864d35861a50 None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:17.698693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb83d86-d143-48ac-ad48-864d35861a50 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:17.700115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb83d86-d143-48ac-ad48-864d35861a50 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:17.783002 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 596/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Sep 19 10:21:17 2022] POST /identity/v3/users => generated 251 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:21:19.970590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e2b62-fcce-4e53-9f0d-b90fd4f7e943 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:19.971109 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e2b62-fcce-4e53-9f0d-b90fd4f7e943 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:19.971221 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e2b62-fcce-4e53-9f0d-b90fd4f7e943 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:19.972512 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 610/1206] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:21:19.977741 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bcf81-fe16-4560-b06b-8d76de9e9b90 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:19.978151 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bcf81-fe16-4560-b06b-8d76de9e9b90 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:19.978306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bcf81-fe16-4560-b06b-8d76de9e9b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:19.996274 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-388bcf81-fe16-4560-b06b-8d76de9e9b90 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:21:19.997507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-388bcf81-fe16-4560-b06b-8d76de9e9b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:21:20.062988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 597/1207] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:21:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:21:20.328449 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e2b62-fcce-4e53-9f0d-b90fd4f7e943 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:20.434685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e2b62-fcce-4e53-9f0d-b90fd4f7e943 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:20.468584 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b23999-110a-44d2-abb9-c27173806c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:20.469225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b23999-110a-44d2-abb9-c27173806c61 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:20.469599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b23999-110a-44d2-abb9-c27173806c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:20.469869 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b23999-110a-44d2-abb9-c27173806c61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:20.479662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b23999-110a-44d2-abb9-c27173806c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:21:20.485050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-f6b23999-110a-44d2-abb9-c27173806c61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:21:20.485050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:20.485050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:20.485050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:20.485050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:20.485050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:20.485050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:20.485050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:20.485050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:20.485050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:20.485050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:20.485050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:20.485050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:21:20.485050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:21:20.485050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:21:20.485050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:21:20.493329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:21:20.493329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:21:20.493329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:21:20.493329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:21:20.493329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:20.493329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:20.493329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:20.493329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:20.493329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:20.493329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:20.493329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:20.493329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:20.493329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:20.493329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:20.493329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:20.493329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:20.494226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:20.494226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:20.494226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:20.494226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:20.494226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:21:20.494226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:21:20.494226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:21:20.494226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:21:20.494226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:21:20.494226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:21:20.494226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:21:20.494226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:20.494226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 611/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:21:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:20.495009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bcf81-fe16-4560-b06b-8d76de9e9b90 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:20.531215 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:20.590114 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bcf81-fe16-4560-b06b-8d76de9e9b90 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:20.624404 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7da29-fd0b-4236-9ed2-e993b6b017a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:20.625154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7da29-fd0b-4236-9ed2-e993b6b017a7 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:20.625439 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7da29-fd0b-4236-9ed2-e993b6b017a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:20.625714 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7da29-fd0b-4236-9ed2-e993b6b017a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:20.626376 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7da29-fd0b-4236-9ed2-e993b6b017a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:20.626727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7da29-fd0b-4236-9ed2-e993b6b017a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:20.628163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7da29-fd0b-4236-9ed2-e993b6b017a7 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:20.635677 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 598/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:21:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:20.642067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b23999-110a-44d2-abb9-c27173806c61 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:20.693294 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:20.742189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b23999-110a-44d2-abb9-c27173806c61 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:20.775320 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b76963-44b7-4291-940a-e6ceace2611f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:20.775980 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b76963-44b7-4291-940a-e6ceace2611f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:20.776260 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b76963-44b7-4291-940a-e6ceace2611f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:20.776533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b76963-44b7-4291-940a-e6ceace2611f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:20.788074 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b76963-44b7-4291-940a-e6ceace2611f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:20.789550 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b76963-44b7-4291-940a-e6ceace2611f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:20.803820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-c2b76963-44b7-4291-940a-e6ceace2611f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:20.803820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:20.803820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:20.803820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:20.803820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:20.803820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:20.803820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:20.803820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:20.803820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:20.803820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:20.803820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:20.803820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:20.803820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:21:20.803820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:21:20.803820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:21:20.803820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:21:20.806169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:20.806169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:20.806169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:20.806169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:20.806169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:20.806169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:20.806169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:20.806169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:20.806169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:20.806169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:20.806169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:20.806169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:20.806169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:20.806169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:20.806169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:20.806169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:20.806169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:21:20.807301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:21:20.807301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:20.807301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:20.807301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 612/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:20.810877 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7da29-fd0b-4236-9ed2-e993b6b017a7 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:20.907085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:20.910043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7da29-fd0b-4236-9ed2-e993b6b017a7 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:20.943657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f11973-a78b-4e41-bf5b-4f3977a0e785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:20.944270 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f11973-a78b-4e41-bf5b-4f3977a0e785 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:20.944549 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f11973-a78b-4e41-bf5b-4f3977a0e785 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:20.944822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f11973-a78b-4e41-bf5b-4f3977a0e785 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:20.945490 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f11973-a78b-4e41-bf5b-4f3977a0e785 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:20.945795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f11973-a78b-4e41-bf5b-4f3977a0e785 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:20.947207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f11973-a78b-4e41-bf5b-4f3977a0e785 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:20.959192 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 599/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:20.965184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b76963-44b7-4291-940a-e6ceace2611f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:21.072429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b76963-44b7-4291-940a-e6ceace2611f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:21.074014 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:21.107067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e9016-fc4c-48c6-ae58-c787787fc9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:21.110015 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e9016-fc4c-48c6-ae58-c787787fc9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:21.110015 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e9016-fc4c-48c6-ae58-c787787fc9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:21.110015 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e9016-fc4c-48c6-ae58-c787787fc9c0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:21.114961 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e9016-fc4c-48c6-ae58-c787787fc9c0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:21.116410 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e9016-fc4c-48c6-ae58-c787787fc9c0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:21.128292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-8f0e9016-fc4c-48c6-ae58-c787787fc9c0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 10:21:21.128292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:21.128292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:21.128292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:21.128292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:21.128292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:21.128292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:21.128292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:21.128292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:21.128292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:21.128292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:21.128292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:21.128292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:21:21.128292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:21:21.128292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:21:21.128292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:21:21.130415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:21.130415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:21.130415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:21:21.130415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:21:21.130415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:21:21.130415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:21:21.130415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:21.130415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:21.130415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:21.130415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:21.130415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:21.130415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:21.130415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:21.130415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:21.130415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:21.130415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:21.130415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:21.130415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:21.131699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:21.131699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:21.131699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:21:21.131699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:21:21.131699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:21:21.131699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:21:21.131699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:21:21.131699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:21:21.131699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 10:21:21.131699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:21.131699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 613/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:21.135399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f11973-a78b-4e41-bf5b-4f3977a0e785 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:21.234901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f11973-a78b-4e41-bf5b-4f3977a0e785 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:21.269246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3353567-1eb6-45ca-9040-0b6a3ecdb882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:21.269840 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3353567-1eb6-45ca-9040-0b6a3ecdb882 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:21.270117 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3353567-1eb6-45ca-9040-0b6a3ecdb882 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:21.270385 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3353567-1eb6-45ca-9040-0b6a3ecdb882 None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:21.271091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3353567-1eb6-45ca-9040-0b6a3ecdb882 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:21.271388 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3353567-1eb6-45ca-9040-0b6a3ecdb882 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:21.272798 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3353567-1eb6-45ca-9040-0b6a3ecdb882 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:21.290693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 600/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Sep 19 10:21:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:21.296755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e9016-fc4c-48c6-ae58-c787787fc9c0 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:21.394747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e9016-fc4c-48c6-ae58-c787787fc9c0 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:21.427012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb46344-37ae-48fc-831d-71e084d1914d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:21.427611 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb46344-37ae-48fc-831d-71e084d1914d None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:21.427892 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb46344-37ae-48fc-831d-71e084d1914d None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:21.428169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb46344-37ae-48fc-831d-71e084d1914d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:21.457691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:21.494239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb46344-37ae-48fc-831d-71e084d1914d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:21.495655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb46344-37ae-48fc-831d-71e084d1914d None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:21.591180 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 614/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:21:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:21:21.597160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3353567-1eb6-45ca-9040-0b6a3ecdb882 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:21.616271 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:21.695707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3353567-1eb6-45ca-9040-0b6a3ecdb882 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:21.730754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9bde3-2c77-4abb-8c62-6130f990fac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:21.731374 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9bde3-2c77-4abb-8c62-6130f990fac4 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:21.731651 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9bde3-2c77-4abb-8c62-6130f990fac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:21.732027 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9bde3-2c77-4abb-8c62-6130f990fac4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:21.742712 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9bde3-2c77-4abb-8c62-6130f990fac4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:21.744153 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9bde3-2c77-4abb-8c62-6130f990fac4 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:21.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-cdc9bde3-2c77-4abb-8c62-6130f990fac4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:21.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:21.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:21.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:21.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:21.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:21.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:21.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:21.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:21.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:21.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:21.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:21.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:21:21.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:21:21.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:21:21.759998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:21:21.761200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:21.761200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:21.761200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:21.761200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:21.761200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:21.761200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:21.761200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:21.761200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:21.761200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:21.761200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:21.761200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:21.761200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:21.761200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:21.761200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:21.761200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:21.761200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:21.761200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:21:21.762299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:21:21.762299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:21.762299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:21:21.762299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 601/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:21.767200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb46344-37ae-48fc-831d-71e084d1914d None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:21.831485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:21.868942 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb46344-37ae-48fc-831d-71e084d1914d None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:21.902107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e652f10-4759-4049-94e2-d5e47a1fb121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:21.902713 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e652f10-4759-4049-94e2-d5e47a1fb121 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:21.902991 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e652f10-4759-4049-94e2-d5e47a1fb121 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:21.903268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e652f10-4759-4049-94e2-d5e47a1fb121 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:21.903904 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e652f10-4759-4049-94e2-d5e47a1fb121 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:21.904208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e652f10-4759-4049-94e2-d5e47a1fb121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:21.906097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e652f10-4759-4049-94e2-d5e47a1fb121 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:21.919292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 615/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:21.924560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9bde3-2c77-4abb-8c62-6130f990fac4 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:21.993412 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:22.023589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9bde3-2c77-4abb-8c62-6130f990fac4 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:22.058774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04253e03-6373-4d31-93d9-37dd4ab5313a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:22.059392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04253e03-6373-4d31-93d9-37dd4ab5313a None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:22.059668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04253e03-6373-4d31-93d9-37dd4ab5313a None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:22.059937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04253e03-6373-4d31-93d9-37dd4ab5313a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:22.066998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04253e03-6373-4d31-93d9-37dd4ab5313a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:22.070365 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04253e03-6373-4d31-93d9-37dd4ab5313a None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:22.079631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-04253e03-6373-4d31-93d9-37dd4ab5313a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:21:22.079631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:22.079631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:22.079631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:22.079631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:22.079631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:22.079631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:22.079631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:22.079631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:22.079631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:22.079631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:22.079631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:22.079631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:21:22.079631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:21:22.079631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:21:22.080670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:21:22.080670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:22.080670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:22.080670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:22.080670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:22.080670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:22.080670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:22.080670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:22.080670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:22.080670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:22.080670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:22.080670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:22.080670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:22.080670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:22.080670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:22.080670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:22.080670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:22.081687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:21:22.081687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:21:22.081687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:21:22.081687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:21:22.081687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:21:22.081687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:21:22.081687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:21:22.081687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:21:22.081687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 602/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:21:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:22.086590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e652f10-4759-4049-94e2-d5e47a1fb121 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:22.184824 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e652f10-4759-4049-94e2-d5e47a1fb121 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:22.206432 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:22.219147 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e92eb-36f4-49d2-b089-84c5c16971ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:22.219747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e92eb-36f4-49d2-b089-84c5c16971ad None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:22.220071 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e92eb-36f4-49d2-b089-84c5c16971ad None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:22.220339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e92eb-36f4-49d2-b089-84c5c16971ad None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:22.221028 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e92eb-36f4-49d2-b089-84c5c16971ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:22.225432 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e92eb-36f4-49d2-b089-84c5c16971ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:22.225432 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e92eb-36f4-49d2-b089-84c5c16971ad None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:22.234581 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 616/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:21:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:22.240736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04253e03-6373-4d31-93d9-37dd4ab5313a None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:22.338298 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04253e03-6373-4d31-93d9-37dd4ab5313a None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:22.365362 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:22.371258 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889d209-223e-4149-818c-2d85c656480d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:22.387090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889d209-223e-4149-818c-2d85c656480d None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:22.387090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889d209-223e-4149-818c-2d85c656480d None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:22.387090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889d209-223e-4149-818c-2d85c656480d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:22.387090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889d209-223e-4149-818c-2d85c656480d None admin] RBAC: Adding query filter params (role.id=39be941a839d4eb6a2146c561c7a2851, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=ce0ef47a60f54ca1853fd267e308c58b) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:22.387090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889d209-223e-4149-818c-2d85c656480d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:22.387090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889d209-223e-4149-818c-2d85c656480d None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:22.387715 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 603/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:21:22 2022] GET /identity/v3/role_assignments?user.id=ce0ef47a60f54ca1853fd267e308c58b&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=39be941a839d4eb6a2146c561c7a2851 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:24.476244 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4054202-598a-4087-8e0f-89bf6d91fb12 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:24.476841 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4054202-598a-4087-8e0f-89bf6d91fb12 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:24.476841 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4054202-598a-4087-8e0f-89bf6d91fb12 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:24.478268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 617/1220] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:21:24.483301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1684865-be72-4b23-84b8-0940c246ccf5 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:24.483768 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1684865-be72-4b23-84b8-0940c246ccf5 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:24.484018 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1684865-be72-4b23-84b8-0940c246ccf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:24.502656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-c1684865-be72-4b23-84b8-0940c246ccf5 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:21:24.503845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c1684865-be72-4b23-84b8-0940c246ccf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:21:24.571160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 604/1221] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:21:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:21:24.842909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4054202-598a-4087-8e0f-89bf6d91fb12 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:24.941549 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4054202-598a-4087-8e0f-89bf6d91fb12 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:24.974067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11682f1d-a5a9-43ff-82b8-0cec5a80baf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:24.974680 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11682f1d-a5a9-43ff-82b8-0cec5a80baf9 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:24.974957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11682f1d-a5a9-43ff-82b8-0cec5a80baf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:24.975231 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11682f1d-a5a9-43ff-82b8-0cec5a80baf9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:24.985675 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11682f1d-a5a9-43ff-82b8-0cec5a80baf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:21:24.990426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-11682f1d-a5a9-43ff-82b8-0cec5a80baf9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:21:24.990426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:24.990426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:24.990426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:24.990426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:24.990426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:24.990426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:24.990426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:24.990426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:24.990426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:24.990426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:24.990426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:24.990426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:21:24.990426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:21:24.990426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:21:24.990426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:21:24.991555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:21:24.991555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:21:24.991555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:21:24.991555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:21:24.991555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:24.991555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:24.991555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:24.991555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:24.991555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:24.991555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:24.991555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:24.991555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:24.991555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:24.991555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:24.991555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:24.991555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:24.992479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:24.992479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:24.992479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:24.992479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:24.992479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:21:24.992479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:21:24.992479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:21:24.992479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:21:24.992479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:21:24.992479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:21:24.992479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:21:24.992479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:24.992479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 618/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:21:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:24.997539 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1684865-be72-4b23-84b8-0940c246ccf5 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:25.094374 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1684865-be72-4b23-84b8-0940c246ccf5 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:25.117707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:25.128266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277f6c0-845d-49fc-aa6c-128e74150805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:25.129985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277f6c0-845d-49fc-aa6c-128e74150805 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:25.129985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277f6c0-845d-49fc-aa6c-128e74150805 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:25.129985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277f6c0-845d-49fc-aa6c-128e74150805 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:25.130361 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d277f6c0-845d-49fc-aa6c-128e74150805 None admin] RBAC: Adding query filter params (name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:25.130361 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d277f6c0-845d-49fc-aa6c-128e74150805 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:25.131718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d277f6c0-845d-49fc-aa6c-128e74150805 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:25.139577 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 605/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:21:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:25.145537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11682f1d-a5a9-43ff-82b8-0cec5a80baf9 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:25.191058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:25.246101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11682f1d-a5a9-43ff-82b8-0cec5a80baf9 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:25.280152 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ce66e-6472-47d4-a9dd-95a27e8fd3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:25.281351 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ce66e-6472-47d4-a9dd-95a27e8fd3ba None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:25.281629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ce66e-6472-47d4-a9dd-95a27e8fd3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:25.281899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ce66e-6472-47d4-a9dd-95a27e8fd3ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:25.292173 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ce66e-6472-47d4-a9dd-95a27e8fd3ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:25.293645 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ce66e-6472-47d4-a9dd-95a27e8fd3ba None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:25.307846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-774ce66e-6472-47d4-a9dd-95a27e8fd3ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:25.307846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:25.307846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:25.307846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:25.307846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:25.307846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:25.307846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:25.307846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:25.307846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:25.307846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:25.307846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:25.307846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:25.307846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:21:25.307846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:21:25.307846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:21:25.307846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:21:25.309000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:25.309000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:25.309000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:25.309000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:25.309000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:25.309000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:25.309000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:25.309000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:25.309000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:25.309000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:25.309000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:25.309000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:25.309000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:25.309000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:25.309000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:25.309000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:25.309000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:21:25.310396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:21:25.310396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:25.310396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:25.310396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 619/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:25.314740 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277f6c0-845d-49fc-aa6c-128e74150805 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:25.414849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277f6c0-845d-49fc-aa6c-128e74150805 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:25.447675 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c7e2c-04da-46b9-946a-862d3f86d30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:25.448302 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c7e2c-04da-46b9-946a-862d3f86d30f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:25.448603 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c7e2c-04da-46b9-946a-862d3f86d30f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:25.448874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c7e2c-04da-46b9-946a-862d3f86d30f None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:25.449623 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c7e2c-04da-46b9-946a-862d3f86d30f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:25.449933 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c7e2c-04da-46b9-946a-862d3f86d30f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:25.451288 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c7e2c-04da-46b9-946a-862d3f86d30f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:25.463173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 606/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:25.469919 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ce66e-6472-47d4-a9dd-95a27e8fd3ba None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:25.561110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:25.567827 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ce66e-6472-47d4-a9dd-95a27e8fd3ba None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:25.603279 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8066e-2fa0-40b9-84a1-be589efa2fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:25.603828 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8066e-2fa0-40b9-84a1-be589efa2fbe None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:25.604106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8066e-2fa0-40b9-84a1-be589efa2fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:25.604376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8066e-2fa0-40b9-84a1-be589efa2fbe None admin] PATH_INFO: `/v3/users/placement` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:25.611208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8066e-2fa0-40b9-84a1-be589efa2fbe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:25.612625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8066e-2fa0-40b9-84a1-be589efa2fbe None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:25.623895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-c1d8066e-2fa0-40b9-84a1-be589efa2fbe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 10:21:25.623895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:25.623895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:25.623895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:25.623895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:25.623895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:25.623895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:25.623895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:25.623895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:25.623895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:25.623895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:25.623895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:25.623895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:21:25.623895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:21:25.623895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:21:25.623895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:21:25.625401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:25.625401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:25.625401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:21:25.625401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:21:25.625401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:21:25.625401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:21:25.625401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:25.625401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:25.625401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:25.625401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:25.625401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:25.625401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:25.625401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:25.625401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:25.625401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:25.625401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:25.625401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:25.625401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:25.626447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:25.626447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:25.626447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:21:25.626447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:21:25.626447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:21:25.626447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:21:25.626447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:21:25.626447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:21:25.626447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 10:21:25.626447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:25.626447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 620/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:25.631480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c7e2c-04da-46b9-946a-862d3f86d30f None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:25.677130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:25.731043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c7e2c-04da-46b9-946a-862d3f86d30f None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:25.764103 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ab166-5852-41a6-84af-088e8673ccd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:25.764762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ab166-5852-41a6-84af-088e8673ccd8 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:25.764762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ab166-5852-41a6-84af-088e8673ccd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:25.766552 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ab166-5852-41a6-84af-088e8673ccd8 None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:25.766773 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ab166-5852-41a6-84af-088e8673ccd8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:25.767073 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ab166-5852-41a6-84af-088e8673ccd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:25.768497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ab166-5852-41a6-84af-088e8673ccd8 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:25.785124 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 607/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Sep 19 10:21:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:25.791007 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8066e-2fa0-40b9-84a1-be589efa2fbe None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:25.891235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8066e-2fa0-40b9-84a1-be589efa2fbe None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:25.923264 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb021c-37fd-4079-bbf9-1130132b5d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:25.923888 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb021c-37fd-4079-bbf9-1130132b5d35 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:25.924165 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb021c-37fd-4079-bbf9-1130132b5d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:25.924436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb021c-37fd-4079-bbf9-1130132b5d35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:25.934644 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb021c-37fd-4079-bbf9-1130132b5d35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:25.936060 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb021c-37fd-4079-bbf9-1130132b5d35 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:25.950420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-3ffb021c-37fd-4079-bbf9-1130132b5d35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:25.950420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:25.950420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:25.950420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:25.950420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:25.950420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:25.950420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:25.950420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:25.950420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:25.950420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:25.950420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:25.950420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:25.950420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:21:25.950420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:21:25.950420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:21:25.950420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:21:25.951609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:25.951609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:25.951609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:25.951609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:25.951609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:25.951609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:25.951609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:25.951609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:25.951609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:25.951609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:25.951609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:25.951609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:25.951609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:25.951609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:25.951609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:25.951609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:25.951609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:21:25.953791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:21:25.953791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:25.953791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:25.953791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 621/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:25.957809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ab166-5852-41a6-84af-088e8673ccd8 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:26.055637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ab166-5852-41a6-84af-088e8673ccd8 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:26.058063 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:26.088877 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06f249-5fee-475c-94ab-20cee8617820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:26.089527 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06f249-5fee-475c-94ab-20cee8617820 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:26.089832 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06f249-5fee-475c-94ab-20cee8617820 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:26.090108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06f249-5fee-475c-94ab-20cee8617820 None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:26.090745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06f249-5fee-475c-94ab-20cee8617820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:26.091049 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06f249-5fee-475c-94ab-20cee8617820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:26.092410 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06f249-5fee-475c-94ab-20cee8617820 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:26.103526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 608/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:26.109829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb021c-37fd-4079-bbf9-1130132b5d35 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:26.119167 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:26.208079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb021c-37fd-4079-bbf9-1130132b5d35 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:26.244308 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e0e50-71c4-47af-8ca6-0e5340bc8810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:26.244925 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e0e50-71c4-47af-8ca6-0e5340bc8810 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:26.245229 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e0e50-71c4-47af-8ca6-0e5340bc8810 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:26.245564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e0e50-71c4-47af-8ca6-0e5340bc8810 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:26.253025 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e0e50-71c4-47af-8ca6-0e5340bc8810 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:26.254491 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e0e50-71c4-47af-8ca6-0e5340bc8810 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:26.266381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-026e0e50-71c4-47af-8ca6-0e5340bc8810 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:21:26.266381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:26.266381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:26.266381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:26.266381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:26.266381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:26.266381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:26.266381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:26.266381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:26.266381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:26.266381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:26.266381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:26.266381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:21:26.266381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:21:26.266381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:21:26.267301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:21:26.267301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:26.267301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:26.267301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:26.267301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:26.267301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:26.267301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:26.267301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:26.267301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:26.267301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:26.267301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:26.267301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:26.267301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:26.267301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:26.267301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:26.267301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:26.267301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:26.268290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:21:26.268290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:21:26.268290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:21:26.268290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:21:26.268290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:21:26.268290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:21:26.268290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:21:26.268290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:26.268290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 622/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:21:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:21:26.273420 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06f249-5fee-475c-94ab-20cee8617820 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:26.372627 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06f249-5fee-475c-94ab-20cee8617820 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:26.407998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47140b3-006d-445b-9fa9-585da5e002f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:26.408602 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47140b3-006d-445b-9fa9-585da5e002f0 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:26.408884 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47140b3-006d-445b-9fa9-585da5e002f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:26.409201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47140b3-006d-445b-9fa9-585da5e002f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:26.409986 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47140b3-006d-445b-9fa9-585da5e002f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:26.414867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47140b3-006d-445b-9fa9-585da5e002f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:26.414867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47140b3-006d-445b-9fa9-585da5e002f0 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:26.424567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 609/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:21:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:26.430790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e0e50-71c4-47af-8ca6-0e5340bc8810 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:26.494788 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:26.527926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e0e50-71c4-47af-8ca6-0e5340bc8810 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:26.561228 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa929e4-04a2-4250-8286-4797efd5ee5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:26.561870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa929e4-04a2-4250-8286-4797efd5ee5e None admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:26.562148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa929e4-04a2-4250-8286-4797efd5ee5e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:26.562420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa929e4-04a2-4250-8286-4797efd5ee5e None admin] PATH_INFO: `/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/ce0ef47a60f54ca1853fd267e308c58b/roles/39be941a839d4eb6a2146c561c7a2851` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:26.576201 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa929e4-04a2-4250-8286-4797efd5ee5e None admin] RBAC: Authorizing `identity:create_grant(project_id=e45b8fbe79994e7585095328a7ec9260, user_id=ce0ef47a60f54ca1853fd267e308c58b, role_id=39be941a839d4eb6a2146c561c7a2851)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:26.577683 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa929e4-04a2-4250-8286-4797efd5ee5e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:26.679132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 623/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:21:26 2022] PUT /identity/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/ce0ef47a60f54ca1853fd267e308c58b/roles/39be941a839d4eb6a2146c561c7a2851 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:21:28.807461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c9660-0991-4abb-b8bc-c4b7032a80dd None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:28.808021 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c9660-0991-4abb-b8bc-c4b7032a80dd None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:28.808021 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c9660-0991-4abb-b8bc-c4b7032a80dd None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:28.809446 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 610/1233] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:21:28.814780 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b563a-e235-4cb5-99da-348979447e33 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:28.815194 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b563a-e235-4cb5-99da-348979447e33 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:28.815329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b563a-e235-4cb5-99da-348979447e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:28.833086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-115b563a-e235-4cb5-99da-348979447e33 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:21:28.834268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-115b563a-e235-4cb5-99da-348979447e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:21:28.899414 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 624/1234] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:21:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:21:29.181143 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c9660-0991-4abb-b8bc-c4b7032a80dd None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:29.206445 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:29.280790 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c9660-0991-4abb-b8bc-c4b7032a80dd None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:29.314251 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf9ce6-9c4c-4ade-aa8d-a190be891c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:29.314876 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf9ce6-9c4c-4ade-aa8d-a190be891c10 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:29.315155 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf9ce6-9c4c-4ade-aa8d-a190be891c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:29.315432 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf9ce6-9c4c-4ade-aa8d-a190be891c10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:29.325639 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf9ce6-9c4c-4ade-aa8d-a190be891c10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:21:29.332918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-06bf9ce6-9c4c-4ade-aa8d-a190be891c10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:21:29.332918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:29.332918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:29.332918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:29.332918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:29.332918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:29.332918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:29.332918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:29.332918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:29.332918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:29.332918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:29.332918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:29.332918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:21:29.332918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:21:29.332918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:21:29.332918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:21:29.334347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:21:29.334347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:21:29.334347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:21:29.334347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:21:29.334347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:29.334347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:29.334347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:29.334347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:29.334347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:29.334347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:29.334347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:29.334347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:29.334347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:29.334347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:29.334347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:29.334347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:29.336186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:29.336186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:29.336186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:29.336186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:29.336186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:21:29.336186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:21:29.336186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:21:29.336186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:21:29.336186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:21:29.336186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:21:29.336186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 10:21:29.336186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:21:29.336186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 611/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Sep 19 10:21:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:29.340952 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b563a-e235-4cb5-99da-348979447e33 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:29.386752 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:29.438233 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b563a-e235-4cb5-99da-348979447e33 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:29.470998 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f37dc-adce-4589-9d85-75f045b5e3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:29.471595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f37dc-adce-4589-9d85-75f045b5e3b5 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:29.471869 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f37dc-adce-4589-9d85-75f045b5e3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:29.472143 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f37dc-adce-4589-9d85-75f045b5e3b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:29.472796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f37dc-adce-4589-9d85-75f045b5e3b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:29.473122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f37dc-adce-4589-9d85-75f045b5e3b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:29.474471 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f37dc-adce-4589-9d85-75f045b5e3b5 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:29.482120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 625/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Sep 19 10:21:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:29.488134 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf9ce6-9c4c-4ade-aa8d-a190be891c10 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:29.573823 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:29.589656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf9ce6-9c4c-4ade-aa8d-a190be891c10 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:29.628642 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d70f5b-db6d-4194-b184-5d4153180fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:29.629753 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d70f5b-db6d-4194-b184-5d4153180fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:29.630196 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d70f5b-db6d-4194-b184-5d4153180fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:29.630472 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d70f5b-db6d-4194-b184-5d4153180fc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:29.641765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d70f5b-db6d-4194-b184-5d4153180fc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:29.643185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d70f5b-db6d-4194-b184-5d4153180fc8 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:29.656945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-55d70f5b-db6d-4194-b184-5d4153180fc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:29.656945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:29.656945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:29.656945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:29.656945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:29.656945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:29.656945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:29.656945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:29.656945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:29.656945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:29.656945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:29.656945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:29.656945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:21:29.656945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:21:29.656945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:21:29.656945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:21:29.659430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:29.659430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:29.659430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:29.659430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:29.659430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:29.659430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:29.659430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:29.659430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:29.659430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:29.659430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:29.659430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:29.659430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:29.659430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:29.659430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:29.659430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:29.659430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:29.659430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:21:29.660338 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:21:29.660338 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:29.660338 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:21:29.660338 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 612/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 10:21:29.664013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f37dc-adce-4589-9d85-75f045b5e3b5 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:29.761772 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f37dc-adce-4589-9d85-75f045b5e3b5 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:29.771079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:29.794173 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee870f-59d7-4fd9-a984-c562a86bc2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:29.794771 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee870f-59d7-4fd9-a984-c562a86bc2b4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:29.795048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee870f-59d7-4fd9-a984-c562a86bc2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:29.795318 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee870f-59d7-4fd9-a984-c562a86bc2b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:29.796893 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee870f-59d7-4fd9-a984-c562a86bc2b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:29.796893 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee870f-59d7-4fd9-a984-c562a86bc2b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:29.798143 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee870f-59d7-4fd9-a984-c562a86bc2b4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:29.809790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 626/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:29.816145 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d70f5b-db6d-4194-b184-5d4153180fc8 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:29.917036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d70f5b-db6d-4194-b184-5d4153180fc8 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:29.950196 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b8f11-9af6-4514-8dfe-cf81580231eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:29.950813 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b8f11-9af6-4514-8dfe-cf81580231eb None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:29.951087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b8f11-9af6-4514-8dfe-cf81580231eb None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:29.951356 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b8f11-9af6-4514-8dfe-cf81580231eb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:29.954045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:29.958937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b8f11-9af6-4514-8dfe-cf81580231eb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:29.960368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b8f11-9af6-4514-8dfe-cf81580231eb None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:29.971606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-ba6b8f11-9af6-4514-8dfe-cf81580231eb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 10:21:29.971606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:29.971606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:29.971606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:29.971606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:29.971606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:29.971606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:29.971606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:29.971606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:29.971606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:29.971606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:29.971606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:29.971606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:21:29.971606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:21:29.971606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:21:29.971606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:21:29.972752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:29.972752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:29.972752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:21:29.972752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:21:29.972752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:21:29.972752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:21:29.972752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:29.972752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:29.972752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:29.972752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:29.972752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:29.972752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:29.972752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:29.972752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:29.972752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:29.972752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:29.972752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:29.972752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:29.973778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:29.973778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:29.973778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:21:29.973778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:21:29.973778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:21:29.973778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:21:29.973778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:21:29.973778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:21:29.973778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 10:21:29.973778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:21:29.973778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 613/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:29.978650 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee870f-59d7-4fd9-a984-c562a86bc2b4 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:30.077332 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee870f-59d7-4fd9-a984-c562a86bc2b4 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:30.111156 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e866cba-0407-4b2e-9874-835c74abff94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:30.111754 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e866cba-0407-4b2e-9874-835c74abff94 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:30.112044 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e866cba-0407-4b2e-9874-835c74abff94 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:30.112377 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e866cba-0407-4b2e-9874-835c74abff94 None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:30.113118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e866cba-0407-4b2e-9874-835c74abff94 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:30.113425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e866cba-0407-4b2e-9874-835c74abff94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:30.114833 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e866cba-0407-4b2e-9874-835c74abff94 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:30.131857 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 627/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Sep 19 10:21:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:30.137736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b8f11-9af6-4514-8dfe-cf81580231eb None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:30.236065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b8f11-9af6-4514-8dfe-cf81580231eb None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:30.269546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c9594-2146-4783-bb51-cd909977bdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:30.270158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c9594-2146-4783-bb51-cd909977bdec None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:30.270435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c9594-2146-4783-bb51-cd909977bdec None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:30.270708 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c9594-2146-4783-bb51-cd909977bdec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:30.330558 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:30.336798 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c9594-2146-4783-bb51-cd909977bdec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:30.338268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c9594-2146-4783-bb51-cd909977bdec None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:30.436628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 614/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:21:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:21:30.442562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e866cba-0407-4b2e-9874-835c74abff94 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:30.469387 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:30.540496 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e866cba-0407-4b2e-9874-835c74abff94 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:30.574318 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37071d0-6d43-4af5-80c9-3de2121f767c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:30.574961 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37071d0-6d43-4af5-80c9-3de2121f767c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:30.575237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37071d0-6d43-4af5-80c9-3de2121f767c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:30.575505 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37071d0-6d43-4af5-80c9-3de2121f767c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:30.587525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37071d0-6d43-4af5-80c9-3de2121f767c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:30.589003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37071d0-6d43-4af5-80c9-3de2121f767c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:30.603359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-d37071d0-6d43-4af5-80c9-3de2121f767c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:30.603359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:30.603359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:30.603359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:30.603359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:30.603359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:30.603359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:30.603359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:30.603359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:30.603359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:30.603359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:30.603359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:30.603359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:21:30.603359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:21:30.603359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:21:30.603359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:21:30.604504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:30.604504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:30.604504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:30.604504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:30.604504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:30.604504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:30.604504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:30.604504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:30.604504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:30.604504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:30.604504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:30.604504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:30.604504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:30.604504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:30.604504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:30.604504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:30.604504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:21:30.605680 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:21:30.605680 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:30.605680 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:30.605680 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 628/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:30.610442 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c9594-2146-4783-bb51-cd909977bdec None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:30.701277 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:30.707455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c9594-2146-4783-bb51-cd909977bdec None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:30.740778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128173d-caaa-4414-8ee6-c78f7c13c84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:30.741402 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128173d-caaa-4414-8ee6-c78f7c13c84c None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:30.741682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128173d-caaa-4414-8ee6-c78f7c13c84c None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:30.741956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128173d-caaa-4414-8ee6-c78f7c13c84c None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:30.742579 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128173d-caaa-4414-8ee6-c78f7c13c84c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:30.742877 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128173d-caaa-4414-8ee6-c78f7c13c84c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:30.744233 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128173d-caaa-4414-8ee6-c78f7c13c84c None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:30.754899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 615/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:30.760612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37071d0-6d43-4af5-80c9-3de2121f767c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:30.839911 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:30.861302 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37071d0-6d43-4af5-80c9-3de2121f767c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:30.894739 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f0a69-ce02-4488-8bde-240a74fcc6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:30.895363 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f0a69-ce02-4488-8bde-240a74fcc6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:30.895642 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f0a69-ce02-4488-8bde-240a74fcc6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:30.895917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f0a69-ce02-4488-8bde-240a74fcc6a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:30.902316 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f0a69-ce02-4488-8bde-240a74fcc6a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:30.903740 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f0a69-ce02-4488-8bde-240a74fcc6a4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:30.914870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-411f0a69-ce02-4488-8bde-240a74fcc6a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:21:30.914870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:30.914870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:30.914870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:30.914870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:30.914870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:30.914870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:30.914870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:30.914870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:30.914870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:30.914870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:30.914870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:30.914870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:21:30.914870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:21:30.914870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:21:30.917860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:21:30.917860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:30.917860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:30.917860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:30.917860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:30.917860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:30.917860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:30.917860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:30.917860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:30.917860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:30.917860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:30.917860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:30.917860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:30.917860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:30.917860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:30.917860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:30.917860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:30.918965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:21:30.918965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:21:30.918965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:21:30.918965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:21:30.918965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:21:30.918965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:21:30.918965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:21:30.918965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:30.918965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 629/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:21:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:30.922041 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128173d-caaa-4414-8ee6-c78f7c13c84c None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:31.022689 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128173d-caaa-4414-8ee6-c78f7c13c84c None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:31.055256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e6085-46df-45aa-a512-1192fa18cf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:31.055855 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e6085-46df-45aa-a512-1192fa18cf98 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:31.056133 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e6085-46df-45aa-a512-1192fa18cf98 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:31.056415 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e6085-46df-45aa-a512-1192fa18cf98 None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:31.057106 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e6085-46df-45aa-a512-1192fa18cf98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:31.057413 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e6085-46df-45aa-a512-1192fa18cf98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:31.058829 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e6085-46df-45aa-a512-1192fa18cf98 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:31.071255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 616/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:21:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:31.077967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f0a69-ce02-4488-8bde-240a74fcc6a4 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:31.173992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f0a69-ce02-4488-8bde-240a74fcc6a4 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:31.208044 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf662a4-0667-46b2-a0d4-92328bcf3432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:31.208656 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf662a4-0667-46b2-a0d4-92328bcf3432 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:31.209008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf662a4-0667-46b2-a0d4-92328bcf3432 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:31.209317 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf662a4-0667-46b2-a0d4-92328bcf3432 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:31.210033 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf662a4-0667-46b2-a0d4-92328bcf3432 None admin] RBAC: Adding query filter params (role.id=39be941a839d4eb6a2146c561c7a2851, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=ce0ef47a60f54ca1853fd267e308c58b) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:31.210339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf662a4-0667-46b2-a0d4-92328bcf3432 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:31.211760 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf662a4-0667-46b2-a0d4-92328bcf3432 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:31.217711 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 630/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:21:31 2022] GET /identity/v3/role_assignments?user.id=ce0ef47a60f54ca1853fd267e308c58b&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=39be941a839d4eb6a2146c561c7a2851 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:33.406660 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59f108-e090-4509-a767-d07b21152af2 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:33.407183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59f108-e090-4509-a767-d07b21152af2 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:33.407183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59f108-e090-4509-a767-d07b21152af2 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:33.408529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 617/1247] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:21:33.413787 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d971d0-0bf7-452d-8e6f-dc94d306d8e3 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:33.414251 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d971d0-0bf7-452d-8e6f-dc94d306d8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:33.414342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d971d0-0bf7-452d-8e6f-dc94d306d8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:33.421151 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:33.433034 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-89d971d0-0bf7-452d-8e6f-dc94d306d8e3 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:21:33.437196 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-89d971d0-0bf7-452d-8e6f-dc94d306d8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:21:33.498268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 631/1248] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:21:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:21:33.772001 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59f108-e090-4509-a767-d07b21152af2 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:33.791672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:33.874728 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59f108-e090-4509-a767-d07b21152af2 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:33.909577 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd0e3c-3d27-4d44-ac7f-7bc62aa1cfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:33.910200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd0e3c-3d27-4d44-ac7f-7bc62aa1cfd8 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:33.910481 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd0e3c-3d27-4d44-ac7f-7bc62aa1cfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:33.910757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd0e3c-3d27-4d44-ac7f-7bc62aa1cfd8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:33.920863 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd0e3c-3d27-4d44-ac7f-7bc62aa1cfd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:21:33.925815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-0cdd0e3c-3d27-4d44-ac7f-7bc62aa1cfd8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:21:33.925815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:33.925815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:33.925815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:33.925815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:33.925815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:33.925815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:33.925815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:33.925815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:33.925815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:33.925815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:33.925815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:33.925815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:21:33.925815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:21:33.925815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:21:33.925815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:21:33.926988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:21:33.926988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:21:33.926988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:21:33.926988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:21:33.926988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:33.926988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:33.926988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:33.926988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:33.926988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:33.926988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:33.926988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:33.926988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:33.926988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:33.926988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:33.926988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:33.926988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:33.928275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:33.928275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:33.928275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:33.928275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:33.928275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:21:33.928275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:21:33.928275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:21:33.928275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:21:33.928275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:21:33.928275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:21:33.928275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:21:33.928275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:21:33.928275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 618/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:21:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:33.933158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d971d0-0bf7-452d-8e6f-dc94d306d8e3 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:34.031701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d971d0-0bf7-452d-8e6f-dc94d306d8e3 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:34.060166 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:34.064980 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc1444-3339-4019-93bf-a6acf66bec46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:34.065610 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc1444-3339-4019-93bf-a6acf66bec46 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:34.065889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc1444-3339-4019-93bf-a6acf66bec46 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:34.066164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc1444-3339-4019-93bf-a6acf66bec46 None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:34.066825 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc1444-3339-4019-93bf-a6acf66bec46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:34.067126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc1444-3339-4019-93bf-a6acf66bec46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:34.068476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc1444-3339-4019-93bf-a6acf66bec46 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:34.076147 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 632/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:21:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:34.082233 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd0e3c-3d27-4d44-ac7f-7bc62aa1cfd8 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:34.159123 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:34.181634 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd0e3c-3d27-4d44-ac7f-7bc62aa1cfd8 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:34.213653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d6ba6-9b30-4755-89f4-aa2e22d185bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:34.214278 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d6ba6-9b30-4755-89f4-aa2e22d185bd None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:34.214563 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d6ba6-9b30-4755-89f4-aa2e22d185bd None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:34.214874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d6ba6-9b30-4755-89f4-aa2e22d185bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:34.226175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d6ba6-9b30-4755-89f4-aa2e22d185bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:34.227630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d6ba6-9b30-4755-89f4-aa2e22d185bd None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:34.242202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-d53d6ba6-9b30-4755-89f4-aa2e22d185bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:34.242202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:34.242202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:34.242202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:34.242202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:34.242202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:34.242202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:34.242202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:34.242202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:34.242202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:34.242202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:34.242202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:34.242202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:21:34.242202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:21:34.242202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:21:34.242202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:21:34.244730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:34.244730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:34.244730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:34.244730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:34.244730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:34.244730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:34.244730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:34.244730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:34.244730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:34.244730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:34.244730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:34.244730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:34.244730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:34.244730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:34.244730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:34.244730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:34.244730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:21:34.245919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:21:34.245919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:34.245919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:21:34.245919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 619/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:34.249253 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc1444-3339-4019-93bf-a6acf66bec46 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:34.349327 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc1444-3339-4019-93bf-a6acf66bec46 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:34.386124 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e76ab-e611-42aa-b372-7d364f98cd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:34.386578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e76ab-e611-42aa-b372-7d364f98cd10 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:34.386856 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e76ab-e611-42aa-b372-7d364f98cd10 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:34.387129 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e76ab-e611-42aa-b372-7d364f98cd10 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:34.387762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e76ab-e611-42aa-b372-7d364f98cd10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:34.388062 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e76ab-e611-42aa-b372-7d364f98cd10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:34.390207 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e76ab-e611-42aa-b372-7d364f98cd10 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:34.402185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 633/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:21:34.408796 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d6ba6-9b30-4755-89f4-aa2e22d185bd None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:34.509793 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d6ba6-9b30-4755-89f4-aa2e22d185bd None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:34.536818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:34.544329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cda165-cff8-41e9-8fe3-7b3480c6ab45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:34.544948 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cda165-cff8-41e9-8fe3-7b3480c6ab45 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:34.545254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cda165-cff8-41e9-8fe3-7b3480c6ab45 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:34.545525 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cda165-cff8-41e9-8fe3-7b3480c6ab45 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:34.553470 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cda165-cff8-41e9-8fe3-7b3480c6ab45 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:34.554896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cda165-cff8-41e9-8fe3-7b3480c6ab45 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:34.565923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-82cda165-cff8-41e9-8fe3-7b3480c6ab45 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 10:21:34.565923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:34.565923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:34.565923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:34.565923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:34.565923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:34.565923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:34.565923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:34.565923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:34.565923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:34.565923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:34.565923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:34.565923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:21:34.565923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:21:34.565923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:21:34.565923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:21:34.569263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:34.569263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:34.569263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:21:34.569263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:21:34.569263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:21:34.569263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:21:34.569263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:34.569263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:34.569263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:34.569263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:34.569263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:34.569263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:34.569263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:34.569263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:34.569263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:34.569263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:34.569263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:34.569263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:34.570227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:34.570227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:34.570227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:21:34.570227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:21:34.570227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:21:34.570227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:21:34.570227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:21:34.570227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:21:34.570227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 10:21:34.570227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:21:34.570227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 620/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:34.573007 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e76ab-e611-42aa-b372-7d364f98cd10 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:34.627679 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:34.670675 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e76ab-e611-42aa-b372-7d364f98cd10 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:34.714729 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fda2da-91af-4191-80ca-f55e98f0a74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:34.715320 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fda2da-91af-4191-80ca-f55e98f0a74e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:34.715594 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fda2da-91af-4191-80ca-f55e98f0a74e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:34.715866 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fda2da-91af-4191-80ca-f55e98f0a74e None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:34.716660 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fda2da-91af-4191-80ca-f55e98f0a74e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:34.717502 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fda2da-91af-4191-80ca-f55e98f0a74e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:34.718971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fda2da-91af-4191-80ca-f55e98f0a74e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:34.736287 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 634/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Sep 19 10:21:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:34.742316 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cda165-cff8-41e9-8fe3-7b3480c6ab45 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:34.842930 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cda165-cff8-41e9-8fe3-7b3480c6ab45 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:34.875804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da304cb-fae2-4b1d-850f-2718eb8910d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:34.876407 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da304cb-fae2-4b1d-850f-2718eb8910d9 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:34.876689 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da304cb-fae2-4b1d-850f-2718eb8910d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:34.876966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da304cb-fae2-4b1d-850f-2718eb8910d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:34.931733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:34.943862 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da304cb-fae2-4b1d-850f-2718eb8910d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:34.945359 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da304cb-fae2-4b1d-850f-2718eb8910d9 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:35.041554 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 621/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:21:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:21:35.047419 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fda2da-91af-4191-80ca-f55e98f0a74e None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:35.146075 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fda2da-91af-4191-80ca-f55e98f0a74e None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:35.159484 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:35.179602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341a294-89f7-4cbe-a8aa-e1c09f56e5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:35.180215 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341a294-89f7-4cbe-a8aa-e1c09f56e5b3 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:35.180494 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341a294-89f7-4cbe-a8aa-e1c09f56e5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:35.180768 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341a294-89f7-4cbe-a8aa-e1c09f56e5b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:35.190535 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341a294-89f7-4cbe-a8aa-e1c09f56e5b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:35.191915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341a294-89f7-4cbe-a8aa-e1c09f56e5b3 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:35.207106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-4341a294-89f7-4cbe-a8aa-e1c09f56e5b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:35.207106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:35.207106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:35.207106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:35.207106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:35.207106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:35.207106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:35.207106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:35.207106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:35.207106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:35.207106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:35.207106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:35.207106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:21:35.207106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:21:35.207106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:21:35.207106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:21:35.208227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:35.208227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:35.208227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:35.208227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:35.208227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:35.208227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:35.208227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:35.208227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:35.208227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:35.208227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:35.208227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:35.208227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:35.208227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:35.208227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:35.208227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:35.208227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:35.208227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:21:35.209543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:21:35.209543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:35.209543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:35.209543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 635/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:35.214008 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da304cb-fae2-4b1d-850f-2718eb8910d9 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:35.300589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:35.312367 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da304cb-fae2-4b1d-850f-2718eb8910d9 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:35.345872 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dcd31c-d959-49e9-a9c0-7cad670a924d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:35.346473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dcd31c-d959-49e9-a9c0-7cad670a924d None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:35.353090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dcd31c-d959-49e9-a9c0-7cad670a924d None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:35.353090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dcd31c-d959-49e9-a9c0-7cad670a924d None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:35.353090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dcd31c-d959-49e9-a9c0-7cad670a924d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:35.353090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dcd31c-d959-49e9-a9c0-7cad670a924d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:35.353090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dcd31c-d959-49e9-a9c0-7cad670a924d None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:35.360159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 622/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:35.365997 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341a294-89f7-4cbe-a8aa-e1c09f56e5b3 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:35.464046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341a294-89f7-4cbe-a8aa-e1c09f56e5b3 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:35.495903 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6a8cf-6310-4dd7-9faf-d4e316d817d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:35.496511 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6a8cf-6310-4dd7-9faf-d4e316d817d0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:35.496788 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6a8cf-6310-4dd7-9faf-d4e316d817d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:35.497135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6a8cf-6310-4dd7-9faf-d4e316d817d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:35.503997 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6a8cf-6310-4dd7-9faf-d4e316d817d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:35.505480 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6a8cf-6310-4dd7-9faf-d4e316d817d0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:35.517448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-03f6a8cf-6310-4dd7-9faf-d4e316d817d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:21:35.517448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:35.517448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:35.517448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:35.517448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:35.517448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:35.517448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:35.517448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:35.517448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:35.517448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:35.517448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:35.517448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:35.517448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:21:35.517448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:21:35.517448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:21:35.518325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:21:35.518325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:35.518325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:35.518325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:35.518325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:35.518325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:35.518325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:35.518325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:35.518325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:35.518325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:35.518325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:35.518325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:35.518325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:35.518325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:35.518325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:35.518325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:35.518325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:35.519289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:21:35.519289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:21:35.519289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:21:35.519289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:21:35.519289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:21:35.519289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:21:35.519289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:21:35.519289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:35.519289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 636/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:21:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:35.524517 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dcd31c-d959-49e9-a9c0-7cad670a924d None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:35.625071 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dcd31c-d959-49e9-a9c0-7cad670a924d None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:35.658554 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b40a9d-cfa2-4d1f-91f5-08bc7610980e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:35.659147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b40a9d-cfa2-4d1f-91f5-08bc7610980e None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:35.659422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b40a9d-cfa2-4d1f-91f5-08bc7610980e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:35.659690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b40a9d-cfa2-4d1f-91f5-08bc7610980e None admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:35.660362 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b40a9d-cfa2-4d1f-91f5-08bc7610980e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:35.660687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b40a9d-cfa2-4d1f-91f5-08bc7610980e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:35.662189 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b40a9d-cfa2-4d1f-91f5-08bc7610980e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:35.673770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 623/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:21:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:35.680003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6a8cf-6310-4dd7-9faf-d4e316d817d0 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:35.706227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:35.785959 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6a8cf-6310-4dd7-9faf-d4e316d817d0 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:35.821853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e8920-f089-437e-9764-1c492a6e8d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:35.822457 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e8920-f089-437e-9764-1c492a6e8d9c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:35.822734 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e8920-f089-437e-9764-1c492a6e8d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:35.823006 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e8920-f089-437e-9764-1c492a6e8d9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:35.823875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e8920-f089-437e-9764-1c492a6e8d9c None admin] RBAC: Adding query filter params (role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=ce0ef47a60f54ca1853fd267e308c58b) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:35.824126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e8920-f089-437e-9764-1c492a6e8d9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:35.825593 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e8920-f089-437e-9764-1c492a6e8d9c None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:35.831032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 637/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:21:35 2022] GET /identity/v3/role_assignments?user.id=ce0ef47a60f54ca1853fd267e308c58b&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:37.973143 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934ce6c-825c-4167-b996-8c84bf11fd71 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:37.973698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934ce6c-825c-4167-b996-8c84bf11fd71 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:37.973698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934ce6c-825c-4167-b996-8c84bf11fd71 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:37.975031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 624/1261] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:21:37.980066 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2d69f-de70-4733-b283-031e3fe75f5e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:37.980459 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2d69f-de70-4733-b283-031e3fe75f5e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:37.980606 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2d69f-de70-4733-b283-031e3fe75f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:37.998914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-15c2d69f-de70-4733-b283-031e3fe75f5e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:21:38.000085 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-15c2d69f-de70-4733-b283-031e3fe75f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:21:38.066654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 638/1262] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:21:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:21:38.339276 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934ce6c-825c-4167-b996-8c84bf11fd71 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:38.350586 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:38.445078 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934ce6c-825c-4167-b996-8c84bf11fd71 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:38.479549 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998ce49-7bd6-4685-9f9f-cbe52ea66f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:38.480179 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998ce49-7bd6-4685-9f9f-cbe52ea66f63 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:38.480459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998ce49-7bd6-4685-9f9f-cbe52ea66f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:38.480734 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998ce49-7bd6-4685-9f9f-cbe52ea66f63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:38.491469 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998ce49-7bd6-4685-9f9f-cbe52ea66f63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:21:38.496255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-2998ce49-7bd6-4685-9f9f-cbe52ea66f63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:21:38.496255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:38.496255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:38.496255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:38.496255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:38.496255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:38.496255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:38.496255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:38.496255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:38.496255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:38.496255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:38.496255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:38.496255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:21:38.496255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:21:38.496255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:21:38.496255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:21:38.497430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:21:38.497430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:21:38.497430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:21:38.497430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:21:38.497430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:38.497430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:38.497430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:38.497430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:38.497430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:38.497430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:38.497430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:38.497430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:38.497430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:38.497430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:38.497430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:38.497430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:38.498328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:38.498328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:38.498328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:38.498328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:38.498328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:21:38.498328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:21:38.498328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:21:38.498328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:21:38.498328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:21:38.498328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:21:38.498328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:21:38.498328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:21:38.498328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 625/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:21:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:38.503650 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2d69f-de70-4733-b283-031e3fe75f5e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:38.521830 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:38.600652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2d69f-de70-4733-b283-031e3fe75f5e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:38.634024 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad3a93-e597-4e53-b6f0-884c8d3f423b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:38.634618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad3a93-e597-4e53-b6f0-884c8d3f423b None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:38.634897 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad3a93-e597-4e53-b6f0-884c8d3f423b None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:38.635218 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad3a93-e597-4e53-b6f0-884c8d3f423b None admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:38.635877 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad3a93-e597-4e53-b6f0-884c8d3f423b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:38.636180 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad3a93-e597-4e53-b6f0-884c8d3f423b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:38.637735 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad3a93-e597-4e53-b6f0-884c8d3f423b None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:38.646005 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 639/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:21:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:38.651851 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998ce49-7bd6-4685-9f9f-cbe52ea66f63 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:38.719396 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:38.752339 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998ce49-7bd6-4685-9f9f-cbe52ea66f63 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:38.786351 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba6c57-2cfd-4d1d-8a1d-7f3452d4d547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:38.786972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba6c57-2cfd-4d1d-8a1d-7f3452d4d547 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:38.787247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba6c57-2cfd-4d1d-8a1d-7f3452d4d547 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:38.787519 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba6c57-2cfd-4d1d-8a1d-7f3452d4d547 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:38.798140 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba6c57-2cfd-4d1d-8a1d-7f3452d4d547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:38.799562 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba6c57-2cfd-4d1d-8a1d-7f3452d4d547 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:38.813955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-0bba6c57-2cfd-4d1d-8a1d-7f3452d4d547 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:38.813955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:38.813955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:38.813955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:38.813955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:38.813955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:38.813955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:38.813955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:38.813955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:38.813955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:38.813955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:38.813955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:38.813955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:21:38.813955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:21:38.813955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:21:38.813955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:21:38.819692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:38.819692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:38.819692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:38.819692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:38.819692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:38.819692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:38.819692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:38.819692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:38.819692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:38.819692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:38.819692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:38.819692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:38.819692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:38.819692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:38.819692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:38.819692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:38.819692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:21:38.820653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:21:38.820653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:38.820653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:21:38.820653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 626/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:38.821244 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad3a93-e597-4e53-b6f0-884c8d3f423b None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:38.901796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:38.921048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad3a93-e597-4e53-b6f0-884c8d3f423b None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:38.954287 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fa01e-73eb-428a-b970-f66b67eed66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:38.954878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fa01e-73eb-428a-b970-f66b67eed66a None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:38.955160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fa01e-73eb-428a-b970-f66b67eed66a None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:38.955439 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fa01e-73eb-428a-b970-f66b67eed66a None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:38.956077 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4fa01e-73eb-428a-b970-f66b67eed66a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:38.956383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4fa01e-73eb-428a-b970-f66b67eed66a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:38.957955 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4fa01e-73eb-428a-b970-f66b67eed66a None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:38.968760 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 640/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:21:38.974516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba6c57-2cfd-4d1d-8a1d-7f3452d4d547 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:39.075473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba6c57-2cfd-4d1d-8a1d-7f3452d4d547 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:39.094825 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:39.109024 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224f3ec-bfa2-4cdf-ac84-dbfa56af5693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:39.109676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224f3ec-bfa2-4cdf-ac84-dbfa56af5693 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:39.109957 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224f3ec-bfa2-4cdf-ac84-dbfa56af5693 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:39.110231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224f3ec-bfa2-4cdf-ac84-dbfa56af5693 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:39.117681 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224f3ec-bfa2-4cdf-ac84-dbfa56af5693 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:39.119117 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224f3ec-bfa2-4cdf-ac84-dbfa56af5693 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:39.130568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-8224f3ec-bfa2-4cdf-ac84-dbfa56af5693 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 10:21:39.130568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:39.130568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:39.130568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:39.130568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:39.130568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:39.130568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:39.130568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:39.130568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:39.130568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:39.130568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:39.130568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:39.130568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:21:39.130568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:21:39.130568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:21:39.130568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:21:39.135482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:39.135482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:39.135482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:21:39.135482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:21:39.135482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:21:39.135482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:21:39.135482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:39.135482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:39.135482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:39.135482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:39.135482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:39.135482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:39.135482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:39.135482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:39.135482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:39.135482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:39.135482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:39.135482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:39.136757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:39.136757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:39.136757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:21:39.136757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:21:39.136757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:21:39.136757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:21:39.136757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:21:39.136757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:21:39.136757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 10:21:39.136757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:21:39.136757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 627/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:39.137636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fa01e-73eb-428a-b970-f66b67eed66a None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:39.236247 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fa01e-73eb-428a-b970-f66b67eed66a None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:39.268721 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba06e8c-f68f-4218-af92-4f6b95d28fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:39.269353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba06e8c-f68f-4218-af92-4f6b95d28fdc None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:39.269634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba06e8c-f68f-4218-af92-4f6b95d28fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:39.269903 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba06e8c-f68f-4218-af92-4f6b95d28fdc None admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:39.270639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba06e8c-f68f-4218-af92-4f6b95d28fdc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:39.270947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba06e8c-f68f-4218-af92-4f6b95d28fdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:39.272367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba06e8c-f68f-4218-af92-4f6b95d28fdc None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:39.282016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:39.290062 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 641/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Sep 19 10:21:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:39.296804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224f3ec-bfa2-4cdf-ac84-dbfa56af5693 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:39.395224 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224f3ec-bfa2-4cdf-ac84-dbfa56af5693 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:39.428665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca470e-5847-4945-a3aa-f023db103a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:39.429315 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca470e-5847-4945-a3aa-f023db103a09 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:39.429595 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca470e-5847-4945-a3aa-f023db103a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:39.429870 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca470e-5847-4945-a3aa-f023db103a09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:39.440083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca470e-5847-4945-a3aa-f023db103a09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:39.441545 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca470e-5847-4945-a3aa-f023db103a09 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:39.455305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-69ca470e-5847-4945-a3aa-f023db103a09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:39.455305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:39.455305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:39.455305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:39.455305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:39.455305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:39.455305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:39.455305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:39.455305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:39.455305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:39.455305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:39.455305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:39.455305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:21:39.455305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:21:39.455305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:21:39.455305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:21:39.461292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:39.461292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:39.461292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:39.461292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:39.461292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:39.461292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:39.461292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:39.461292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:39.461292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:39.461292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:39.461292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:39.461292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:39.461292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:39.461292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:39.461292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:39.461292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:39.461292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:21:39.462386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:21:39.462386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:39.462386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:21:39.462386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 628/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:39.462626 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba06e8c-f68f-4218-af92-4f6b95d28fdc None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:39.561474 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba06e8c-f68f-4218-af92-4f6b95d28fdc None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:39.595279 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed5461-8ee6-4256-902b-9db302f726c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:39.595879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed5461-8ee6-4256-902b-9db302f726c9 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:39.596158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed5461-8ee6-4256-902b-9db302f726c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:39.596429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed5461-8ee6-4256-902b-9db302f726c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:39.597122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed5461-8ee6-4256-902b-9db302f726c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:39.597455 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed5461-8ee6-4256-902b-9db302f726c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:39.598796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed5461-8ee6-4256-902b-9db302f726c9 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:39.610347 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 642/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:39.616039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca470e-5847-4945-a3aa-f023db103a09 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:39.648237 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:39.715534 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca470e-5847-4945-a3aa-f023db103a09 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:39.749698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff501ea9-4519-49e9-be61-ad4bfdd450a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:39.750320 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff501ea9-4519-49e9-be61-ad4bfdd450a5 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:39.750603 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff501ea9-4519-49e9-be61-ad4bfdd450a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:39.750886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff501ea9-4519-49e9-be61-ad4bfdd450a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:39.757597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff501ea9-4519-49e9-be61-ad4bfdd450a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:39.759239 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff501ea9-4519-49e9-be61-ad4bfdd450a5 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:39.771756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-ff501ea9-4519-49e9-be61-ad4bfdd450a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:21:39.771756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:39.771756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:39.771756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:39.771756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:39.771756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:39.771756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:39.771756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:39.771756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:39.771756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:39.771756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:39.771756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:39.771756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:21:39.771756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:21:39.771756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:21:39.772658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:21:39.772658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:39.772658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:39.772658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:39.772658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:39.772658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:39.772658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:39.772658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:39.772658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:39.772658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:39.772658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:39.772658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:39.772658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:39.772658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:39.772658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:39.772658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:39.772658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:39.773795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:21:39.773795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:21:39.773795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:21:39.773795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:21:39.773795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:21:39.773795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:21:39.773795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:21:39.773795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:21:39.773795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 629/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:21:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:39.779009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed5461-8ee6-4256-902b-9db302f726c9 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:39.837322 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:39.877822 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed5461-8ee6-4256-902b-9db302f726c9 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:39.910901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506872e0-0339-49b2-99d0-aad88dda21c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:39.911525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506872e0-0339-49b2-99d0-aad88dda21c2 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:39.911805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506872e0-0339-49b2-99d0-aad88dda21c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:39.912080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506872e0-0339-49b2-99d0-aad88dda21c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:39.912761 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506872e0-0339-49b2-99d0-aad88dda21c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:39.913090 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506872e0-0339-49b2-99d0-aad88dda21c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:39.914504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506872e0-0339-49b2-99d0-aad88dda21c2 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:39.926715 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 643/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:21:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:39.932839 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff501ea9-4519-49e9-be61-ad4bfdd450a5 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:40.022133 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:40.031025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff501ea9-4519-49e9-be61-ad4bfdd450a5 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:40.064758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c3755-6946-443d-9d3b-ee3ddfb0ce20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:40.065528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c3755-6946-443d-9d3b-ee3ddfb0ce20 None admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:40.065818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c3755-6946-443d-9d3b-ee3ddfb0ce20 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:40.066080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c3755-6946-443d-9d3b-ee3ddfb0ce20 None admin] PATH_INFO: `/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/ce0ef47a60f54ca1853fd267e308c58b/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:40.068043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c3755-6946-443d-9d3b-ee3ddfb0ce20 None admin] RBAC: Authorizing `identity:create_grant(project_id=e45b8fbe79994e7585095328a7ec9260, user_id=ce0ef47a60f54ca1853fd267e308c58b, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:40.069492 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c3755-6946-443d-9d3b-ee3ddfb0ce20 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:40.203922 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 630/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Sep 19 10:21:39 2022] PUT /identity/v3/projects/e45b8fbe79994e7585095328a7ec9260/users/ce0ef47a60f54ca1853fd267e308c58b/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:21:42.360286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff2e97-b3d1-40a9-8899-6e5c468c7241 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:42.360839 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff2e97-b3d1-40a9-8899-6e5c468c7241 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:42.360839 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff2e97-b3d1-40a9-8899-6e5c468c7241 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:42.362242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 644/1274] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:21:42.367740 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5143a-79ef-47ed-8d55-70fa87db332c None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:42.368135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5143a-79ef-47ed-8d55-70fa87db332c None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:42.372404 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5143a-79ef-47ed-8d55-70fa87db332c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:42.386347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-2fd5143a-79ef-47ed-8d55-70fa87db332c None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:21:42.387547 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-2fd5143a-79ef-47ed-8d55-70fa87db332c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:21:42.457649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 631/1275] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:21:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:21:42.726340 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff2e97-b3d1-40a9-8899-6e5c468c7241 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:42.826272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff2e97-b3d1-40a9-8899-6e5c468c7241 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:42.843843 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:42.859710 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c5676-3f24-43e0-83e8-6f3cf0c62e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:42.860329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c5676-3f24-43e0-83e8-6f3cf0c62e5c None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:42.860609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c5676-3f24-43e0-83e8-6f3cf0c62e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:42.860881 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c5676-3f24-43e0-83e8-6f3cf0c62e5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:42.870480 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c5676-3f24-43e0-83e8-6f3cf0c62e5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 10:21:42.875797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-511c5676-3f24-43e0-83e8-6f3cf0c62e5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:21:42.875797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:42.875797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:42.875797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:42.875797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:42.875797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:42.875797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:42.875797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:42.875797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:42.875797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:42.875797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:42.875797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:42.875797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 10:21:42.875797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 10:21:42.875797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 10:21:42.875797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 10:21:42.876910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 10:21:42.876910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 10:21:42.876910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 10:21:42.876910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 10:21:42.876910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:42.876910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:42.876910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:42.876910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:42.876910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:42.876910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:42.876910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:42.876910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:42.876910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:42.876910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:42.876910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:42.876910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:42.877814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:42.877814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:42.877814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:42.877814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:42.877814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 10:21:42.877814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 10:21:42.877814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 10:21:42.877814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 10:21:42.877814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 10:21:42.877814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 10:21:42.877814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 10:21:42.877814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:42.877814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 645/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Sep 19 10:21:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:42.883237 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5143a-79ef-47ed-8d55-70fa87db332c None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:42.962973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:42.980398 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5143a-79ef-47ed-8d55-70fa87db332c None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:43.013013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cc75c-7ea3-4a20-ab35-a48e32b63c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:43.014556 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cc75c-7ea3-4a20-ab35-a48e32b63c93 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:43.014830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cc75c-7ea3-4a20-ab35-a48e32b63c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:43.015100 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cc75c-7ea3-4a20-ab35-a48e32b63c93 None admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:43.015753 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91cc75c-7ea3-4a20-ab35-a48e32b63c93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:43.016071 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91cc75c-7ea3-4a20-ab35-a48e32b63c93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:43.017438 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91cc75c-7ea3-4a20-ab35-a48e32b63c93 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:43.025120 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 632/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Sep 19 10:21:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:43.031141 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c5676-3f24-43e0-83e8-6f3cf0c62e5c None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:43.132492 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c5676-3f24-43e0-83e8-6f3cf0c62e5c None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:43.165498 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f881ee2-6b31-4738-9377-747acd674f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:43.166113 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f881ee2-6b31-4738-9377-747acd674f68 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:43.166394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f881ee2-6b31-4738-9377-747acd674f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:43.166670 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f881ee2-6b31-4738-9377-747acd674f68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:43.176309 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f881ee2-6b31-4738-9377-747acd674f68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:43.177774 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f881ee2-6b31-4738-9377-747acd674f68 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:43.192383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-6f881ee2-6b31-4738-9377-747acd674f68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:43.192383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:43.192383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:43.192383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:43.192383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:43.192383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:43.192383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:43.192383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:43.192383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:43.192383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:43.192383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:43.192383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:43.192383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:21:43.192383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:21:43.192383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:21:43.192383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:21:43.193563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:43.193563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:43.193563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:43.193563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:43.193563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:43.193563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:43.193563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:43.193563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:43.193563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:43.193563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:43.193563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:43.193563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:43.193563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:43.193563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:43.193563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:43.193563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:43.193563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:21:43.194698 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:21:43.194698 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:43.194698 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:43.194698 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 646/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:43.199477 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cc75c-7ea3-4a20-ab35-a48e32b63c93 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:43.298375 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cc75c-7ea3-4a20-ab35-a48e32b63c93 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:43.331247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2339007-5aab-4fcf-876b-8e8d2c798a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:43.331891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2339007-5aab-4fcf-876b-8e8d2c798a1f None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:43.332169 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2339007-5aab-4fcf-876b-8e8d2c798a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:43.332444 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2339007-5aab-4fcf-876b-8e8d2c798a1f None admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:43.333138 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2339007-5aab-4fcf-876b-8e8d2c798a1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:43.333439 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2339007-5aab-4fcf-876b-8e8d2c798a1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:43.334849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2339007-5aab-4fcf-876b-8e8d2c798a1f None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:43.346434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 633/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:43.352175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f881ee2-6b31-4738-9377-747acd674f68 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:43.383302 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:43.449709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f881ee2-6b31-4738-9377-747acd674f68 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:43.483372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c16fd-01e6-4e99-9a41-c815d3877d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:43.483980 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c16fd-01e6-4e99-9a41-c815d3877d47 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:43.484255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c16fd-01e6-4e99-9a41-c815d3877d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:43.484525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c16fd-01e6-4e99-9a41-c815d3877d47 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:43.491208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c16fd-01e6-4e99-9a41-c815d3877d47 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:43.492630 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c16fd-01e6-4e99-9a41-c815d3877d47 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:43.504537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-0f4c16fd-01e6-4e99-9a41-c815d3877d47 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 10:21:43.504537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:43.504537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:43.504537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:43.504537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:43.504537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:43.504537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:43.504537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:43.504537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:43.504537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:43.504537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:43.504537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:43.504537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 10:21:43.504537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 10:21:43.504537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 10:21:43.504537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 10:21:43.506521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:43.506521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:43.506521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 10:21:43.506521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:21:43.506521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 10:21:43.506521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 10:21:43.506521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:43.506521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:43.506521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:43.506521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:43.506521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:43.506521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:43.506521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:43.506521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:43.506521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:43.506521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:43.506521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:43.506521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:43.507746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:43.507746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:43.507746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 10:21:43.507746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 10:21:43.507746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 10:21:43.507746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 10:21:43.507746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 10:21:43.507746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 10:21:43.507746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 10:21:43.507746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:43.507746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 647/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:43.511712 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2339007-5aab-4fcf-876b-8e8d2c798a1f None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:43.515796 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:43.609104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2339007-5aab-4fcf-876b-8e8d2c798a1f None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:43.643093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a4049-abdc-4c54-9137-7fb97baf7353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:43.643695 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a4049-abdc-4c54-9137-7fb97baf7353 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:43.643977 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a4049-abdc-4c54-9137-7fb97baf7353 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:43.644249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a4049-abdc-4c54-9137-7fb97baf7353 None admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:43.644962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a4049-abdc-4c54-9137-7fb97baf7353 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:43.645302 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a4049-abdc-4c54-9137-7fb97baf7353 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:43.646726 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a4049-abdc-4c54-9137-7fb97baf7353 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:43.664597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 634/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Sep 19 10:21:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:43.671012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c16fd-01e6-4e99-9a41-c815d3877d47 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:43.757326 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:43.766689 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c16fd-01e6-4e99-9a41-c815d3877d47 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:43.799216 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b978f-12a9-47d4-b0ff-7e4177c7de9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:43.799810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b978f-12a9-47d4-b0ff-7e4177c7de9f None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:43.800089 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b978f-12a9-47d4-b0ff-7e4177c7de9f None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:43.800364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b978f-12a9-47d4-b0ff-7e4177c7de9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:43.866395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b978f-12a9-47d4-b0ff-7e4177c7de9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:43.867774 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b978f-12a9-47d4-b0ff-7e4177c7de9f None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:43.954059 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:43.964117 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 648/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Sep 19 10:21:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 10:21:43.970631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a4049-abdc-4c54-9137-7fb97baf7353 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:44.030589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:44.066488 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a4049-abdc-4c54-9137-7fb97baf7353 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:44.101243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389f8a5-e2fd-443c-8ac4-df969b410cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:44.101871 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389f8a5-e2fd-443c-8ac4-df969b410cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:44.102163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389f8a5-e2fd-443c-8ac4-df969b410cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:44.102425 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389f8a5-e2fd-443c-8ac4-df969b410cf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:44.114211 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389f8a5-e2fd-443c-8ac4-df969b410cf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:44.115676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389f8a5-e2fd-443c-8ac4-df969b410cf1 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:44.130057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-3389f8a5-e2fd-443c-8ac4-df969b410cf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:44.130057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:44.130057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:44.130057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:44.130057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:44.130057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:44.130057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:44.130057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:44.130057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:44.130057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:44.130057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:44.130057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:44.130057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 10:21:44.130057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 10:21:44.130057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 10:21:44.130057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 10:21:44.131036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:44.131036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:44.131036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:44.131036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:44.131036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:44.131036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:44.131036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:44.131036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:44.131036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:44.131036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:44.131036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:44.131036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:44.131036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:44.131036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:44.131036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:44.131036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:44.131036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 10:21:44.132051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 10:21:44.132051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 10:21:44.132051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:21:44.132051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 635/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:21:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:44.137095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b978f-12a9-47d4-b0ff-7e4177c7de9f None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:44.233893 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b978f-12a9-47d4-b0ff-7e4177c7de9f None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:44.267050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf05234-4a1e-44af-9ccb-894d9d526be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:44.267633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf05234-4a1e-44af-9ccb-894d9d526be4 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:44.267908 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf05234-4a1e-44af-9ccb-894d9d526be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:44.268179 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf05234-4a1e-44af-9ccb-894d9d526be4 None admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:44.268801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf05234-4a1e-44af-9ccb-894d9d526be4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:44.269120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf05234-4a1e-44af-9ccb-894d9d526be4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:44.270457 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf05234-4a1e-44af-9ccb-894d9d526be4 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:44.281869 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 649/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Sep 19 10:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:44.287750 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389f8a5-e2fd-443c-8ac4-df969b410cf1 None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:44.388111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389f8a5-e2fd-443c-8ac4-df969b410cf1 None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:44.402093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:44.421647 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0ab3d-99a0-4506-9188-4987d156c6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:44.422269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0ab3d-99a0-4506-9188-4987d156c6ba None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:44.422547 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0ab3d-99a0-4506-9188-4987d156c6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:44.422817 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0ab3d-99a0-4506-9188-4987d156c6ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:44.429287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0ab3d-99a0-4506-9188-4987d156c6ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:44.430718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0ab3d-99a0-4506-9188-4987d156c6ba None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:44.441984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application [None req-52a0ab3d-99a0-4506-9188-4987d156c6ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:21:44.441984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:44.441984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:44.441984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:44.441984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:44.441984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:44.441984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:44.441984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:44.441984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:44.441984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:44.441984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:44.441984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:44.441984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 10:21:44.441984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 10:21:44.441984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 10:21:44.443092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 10:21:44.443092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:44.443092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:44.443092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:44.443092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:44.443092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:44.443092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:44.443092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:44.443092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:44.443092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:44.443092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:44.443092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:44.443092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:44.443092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:44.443092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:44.443092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:44.443092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:44.444200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 10:21:44.444200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 10:21:44.444200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 10:21:44.444200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 10:21:44.444200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 10:21:44.444200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 10:21:44.444200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 10:21:44.444200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: ERROR keystone.server.flask.application Sep 19 10:21:44.444200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 636/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Sep 19 10:21:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:44.449235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf05234-4a1e-44af-9ccb-894d9d526be4 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:44.500529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:44.548913 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf05234-4a1e-44af-9ccb-894d9d526be4 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:44.581360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3576fafc-ffa4-4bb6-b808-5bf29c6a0143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:44.581955 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576fafc-ffa4-4bb6-b808-5bf29c6a0143 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:44.582230 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576fafc-ffa4-4bb6-b808-5bf29c6a0143 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:44.582503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576fafc-ffa4-4bb6-b808-5bf29c6a0143 None admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:44.583193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3576fafc-ffa4-4bb6-b808-5bf29c6a0143 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:44.583530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3576fafc-ffa4-4bb6-b808-5bf29c6a0143 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:44.584939 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3576fafc-ffa4-4bb6-b808-5bf29c6a0143 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:44.597676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 650/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Sep 19 10:21:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:21:44.603938 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0ab3d-99a0-4506-9188-4987d156c6ba None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:44.702425 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0ab3d-99a0-4506-9188-4987d156c6ba None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:44.734974 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00acfc-2d48-4407-bf84-3d9bf28156c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:44.735571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00acfc-2d48-4407-bf84-3d9bf28156c6 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:44.735848 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00acfc-2d48-4407-bf84-3d9bf28156c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:44.736123 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00acfc-2d48-4407-bf84-3d9bf28156c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:44.736833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00acfc-2d48-4407-bf84-3d9bf28156c6 None admin] RBAC: Adding query filter params (role.id=1135c6b2ed2f4ba290fbb4a363456a49, scope.project.id=e45b8fbe79994e7585095328a7ec9260, user.id=ce0ef47a60f54ca1853fd267e308c58b) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:44.737164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00acfc-2d48-4407-bf84-3d9bf28156c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:44.738599 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00acfc-2d48-4407-bf84-3d9bf28156c6 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:44.744147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 637/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Sep 19 10:21:44 2022] GET /identity/v3/role_assignments?user.id=ce0ef47a60f54ca1853fd267e308c58b&scope.project.id=e45b8fbe79994e7585095328a7ec9260&role.id=1135c6b2ed2f4ba290fbb4a363456a49 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:46.920097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ba3d2-ba82-4251-8309-bbfa46d08f2f None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:46.920681 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ba3d2-ba82-4251-8309-bbfa46d08f2f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:46.920681 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ba3d2-ba82-4251-8309-bbfa46d08f2f None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:46.922029 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 651/1288] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:21:46.927208 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919723ec-69b5-437a-9825-287febe94066 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:46.927611 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919723ec-69b5-437a-9825-287febe94066 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:46.927756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919723ec-69b5-437a-9825-287febe94066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:46.946029 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-919723ec-69b5-437a-9825-287febe94066 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:21:46.947201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-919723ec-69b5-437a-9825-287febe94066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:21:46.976059 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:47.012691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 638/1289] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:21:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:21:47.286880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ba3d2-ba82-4251-8309-bbfa46d08f2f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:47.387200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ba3d2-ba82-4251-8309-bbfa46d08f2f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:47.398072 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:47.421338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ba8d9-7c47-44a0-8593-8d0e74b30266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:47.421953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ba8d9-7c47-44a0-8593-8d0e74b30266 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:47.422236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ba8d9-7c47-44a0-8593-8d0e74b30266 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:47.422509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ba8d9-7c47-44a0-8593-8d0e74b30266 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:47.423082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ba8d9-7c47-44a0-8593-8d0e74b30266 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:47.424433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ba8d9-7c47-44a0-8593-8d0e74b30266 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:47.432850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-9c9ba8d9-7c47-44a0-8593-8d0e74b30266 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 10:21:47.432850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:47.432850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:21:47.432850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:21:47.432850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:21:47.432850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:21:47.432850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:21:47.432850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:21:47.432850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 10:21:47.432850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:47.432850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:21:47.432850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:47.432850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:47.432850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:47.432850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:47.432850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:47.432850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:47.434108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:47.434108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:47.434108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:47.434108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:47.434108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:47.434108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:47.434108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:21:47.434108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:21:47.434108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:21:47.434108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:21:47.434108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:47.434108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:47.434108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:47.434108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:47.434108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:47.434108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:47.435008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:47.435008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:47.435008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:47.435008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:47.435008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:47.435008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:47.435008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:47.435008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:47.435008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:47.435008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:47.435008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:21:47.435008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:21:47.435008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 10:21:47.435008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:47.435008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 652/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 19 10:21:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:47.440015 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919723ec-69b5-437a-9825-287febe94066 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:47.536384 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919723ec-69b5-437a-9825-287febe94066 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:47.571239 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ca5c8-5162-409f-9d68-0e77b6e7a586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:47.571779 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ca5c8-5162-409f-9d68-0e77b6e7a586 None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:47.572059 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ca5c8-5162-409f-9d68-0e77b6e7a586 None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:47.572332 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ca5c8-5162-409f-9d68-0e77b6e7a586 None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:47.572960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ca5c8-5162-409f-9d68-0e77b6e7a586 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:47.573296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ca5c8-5162-409f-9d68-0e77b6e7a586 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:47.574667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ca5c8-5162-409f-9d68-0e77b6e7a586 None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:47.579459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 639/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Sep 19 10:21:47 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:47.585170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ba8d9-7c47-44a0-8593-8d0e74b30266 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:47.684873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ba8d9-7c47-44a0-8593-8d0e74b30266 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:47.718977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db746b6a-2aef-4b19-ad79-f001e1fe6b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:47.719582 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db746b6a-2aef-4b19-ad79-f001e1fe6b4e None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:47.719861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db746b6a-2aef-4b19-ad79-f001e1fe6b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:47.720137 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db746b6a-2aef-4b19-ad79-f001e1fe6b4e None admin] PATH_INFO: `/v3/services` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:47.720762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db746b6a-2aef-4b19-ad79-f001e1fe6b4e None admin] RBAC: Adding query filter params (type=placement) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:47.721084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db746b6a-2aef-4b19-ad79-f001e1fe6b4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:47.722451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db746b6a-2aef-4b19-ad79-f001e1fe6b4e None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:47.726803 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 653/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Sep 19 10:21:47 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:49.866304 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5093235-a7a4-454d-ad51-01db1d22b041 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:49.866767 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5093235-a7a4-454d-ad51-01db1d22b041 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:49.866888 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5093235-a7a4-454d-ad51-01db1d22b041 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:49.868148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 640/1293] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:21:49.873875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010b548-bd33-40f5-9042-07027c57efaf None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:49.874276 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010b548-bd33-40f5-9042-07027c57efaf None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:49.874437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010b548-bd33-40f5-9042-07027c57efaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:49.893179 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1010b548-bd33-40f5-9042-07027c57efaf None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:21:49.894373 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1010b548-bd33-40f5-9042-07027c57efaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:21:49.922643 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:49.959080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 654/1294] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:21:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:21:50.232126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5093235-a7a4-454d-ad51-01db1d22b041 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:50.262547 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:50.331638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5093235-a7a4-454d-ad51-01db1d22b041 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:50.365760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16578a60-0f27-4a44-a9d6-7ee12495487e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:50.366355 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16578a60-0f27-4a44-a9d6-7ee12495487e None admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:50.366630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16578a60-0f27-4a44-a9d6-7ee12495487e None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:50.366898 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16578a60-0f27-4a44-a9d6-7ee12495487e None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:50.367533 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16578a60-0f27-4a44-a9d6-7ee12495487e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:50.368946 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16578a60-0f27-4a44-a9d6-7ee12495487e None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:50.432423 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 641/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Sep 19 10:21:50 2022] POST /identity/v3/services => generated 246 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:21:52.593775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359720f1-5a98-4e22-a064-2a6e857d4905 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:52.594337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359720f1-5a98-4e22-a064-2a6e857d4905 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:52.594337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359720f1-5a98-4e22-a064-2a6e857d4905 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:52.595882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 655/1296] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:21:52.600973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a39190-7053-49c0-affe-b047d23f9795 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:52.601431 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a39190-7053-49c0-affe-b047d23f9795 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:52.601551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a39190-7053-49c0-affe-b047d23f9795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:52.619436 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-22a39190-7053-49c0-affe-b047d23f9795 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:21:52.620609 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-22a39190-7053-49c0-affe-b047d23f9795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:21:52.691250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 642/1297] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:21:52 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:21:52.965963 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359720f1-5a98-4e22-a064-2a6e857d4905 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:53.065917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359720f1-5a98-4e22-a064-2a6e857d4905 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:53.098671 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b006767-5866-4780-b44f-a8c26c6ccd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:53.099340 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b006767-5866-4780-b44f-a8c26c6ccd07 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:53.099681 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b006767-5866-4780-b44f-a8c26c6ccd07 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:53.099937 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b006767-5866-4780-b44f-a8c26c6ccd07 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:53.100507 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b006767-5866-4780-b44f-a8c26c6ccd07 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:53.101892 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b006767-5866-4780-b44f-a8c26c6ccd07 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:53.110902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-4b006767-5866-4780-b44f-a8c26c6ccd07 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 10:21:53.110902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:53.110902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:21:53.110902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:21:53.110902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:21:53.110902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:21:53.110902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:21:53.110902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:21:53.110902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 10:21:53.110902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:53.110902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:21:53.110902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:53.110902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:53.110902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:53.110902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:53.110902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:53.110902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:53.112120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:53.112120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:53.112120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:53.112120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:53.112120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:53.112120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:53.112120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:21:53.112120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:21:53.112120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:21:53.112120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:21:53.112120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:53.112120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:53.112120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:53.112120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:53.112120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:53.112120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:53.112988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:53.112988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:53.112988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:53.112988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:53.112988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:53.112988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:53.112988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:53.112988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:53.112988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:53.112988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:53.112988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:21:53.112988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:21:53.112988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 10:21:53.112988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:53.112988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 656/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 19 10:21:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:53.118112 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a39190-7053-49c0-affe-b047d23f9795 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:53.136530 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:53.213192 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a39190-7053-49c0-affe-b047d23f9795 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:53.247126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8790e0-b0e7-44a3-a460-99b6d92abdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:53.247660 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8790e0-b0e7-44a3-a460-99b6d92abdcd None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:53.247992 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8790e0-b0e7-44a3-a460-99b6d92abdcd None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:53.248272 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8790e0-b0e7-44a3-a460-99b6d92abdcd None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:53.248913 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8790e0-b0e7-44a3-a460-99b6d92abdcd None admin] RBAC: Adding query filter params (name=placement) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:53.249252 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8790e0-b0e7-44a3-a460-99b6d92abdcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:53.250624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8790e0-b0e7-44a3-a460-99b6d92abdcd None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:53.255793 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 643/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Sep 19 10:21:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:21:53.261860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b006767-5866-4780-b44f-a8c26c6ccd07 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:53.294249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:53.360377 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b006767-5866-4780-b44f-a8c26c6ccd07 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:53.394462 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7593aca-d7f3-47ce-9d95-9082d0f84465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:53.395054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7593aca-d7f3-47ce-9d95-9082d0f84465 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:53.395329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7593aca-d7f3-47ce-9d95-9082d0f84465 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:53.395597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7593aca-d7f3-47ce-9d95-9082d0f84465 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:53.396314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7593aca-d7f3-47ce-9d95-9082d0f84465 None admin] RBAC: Adding query filter params (interface=public, service_id=cef8911fe6e44e45b20433468fcf2170, region_id=RegionOne) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:53.396630 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7593aca-d7f3-47ce-9d95-9082d0f84465 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:53.397999 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7593aca-d7f3-47ce-9d95-9082d0f84465 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:53.403447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 657/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Sep 19 10:21:53 2022] GET /identity/v3/endpoints?service_id=cef8911fe6e44e45b20433468fcf2170&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:53.409269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8790e0-b0e7-44a3-a460-99b6d92abdcd None admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:53.490205 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:53.508281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8790e0-b0e7-44a3-a460-99b6d92abdcd None admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:53.541942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43639304-25d3-4ae3-9773-81010478518b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:53.542540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43639304-25d3-4ae3-9773-81010478518b None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:53.542820 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43639304-25d3-4ae3-9773-81010478518b None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:53.543094 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43639304-25d3-4ae3-9773-81010478518b None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:53.543692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43639304-25d3-4ae3-9773-81010478518b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:53.545085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43639304-25d3-4ae3-9773-81010478518b None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:53.551759 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 644/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Sep 19 10:21:53 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:21:55.682561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b6352-9474-4988-9f31-e63109d86bab None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:55.683223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b6352-9474-4988-9f31-e63109d86bab None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:55.683223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b6352-9474-4988-9f31-e63109d86bab None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:55.684964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 658/1302] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:21:55.690275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299a0a6-0c2d-4e38-9591-02d45676712b None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:55.690706 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299a0a6-0c2d-4e38-9591-02d45676712b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:55.690875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299a0a6-0c2d-4e38-9591-02d45676712b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:55.709451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-d299a0a6-0c2d-4e38-9591-02d45676712b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:21:55.710678 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d299a0a6-0c2d-4e38-9591-02d45676712b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:21:55.775138 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 645/1303] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:21:55 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 10:21:56.041564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b6352-9474-4988-9f31-e63109d86bab None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:56.134095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:56.140834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b6352-9474-4988-9f31-e63109d86bab None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:56.173207 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc50a3a-3008-4878-8d3e-54f6a48cb2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:56.173825 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc50a3a-3008-4878-8d3e-54f6a48cb2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:56.174104 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc50a3a-3008-4878-8d3e-54f6a48cb2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:56.174377 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc50a3a-3008-4878-8d3e-54f6a48cb2b0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:56.174953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc50a3a-3008-4878-8d3e-54f6a48cb2b0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:56.176305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc50a3a-3008-4878-8d3e-54f6a48cb2b0 None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:56.185366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application [None req-3cc50a3a-3008-4878-8d3e-54f6a48cb2b0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 10:21:56.185366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:56.185366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 10:21:56.185366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 10:21:56.185366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 10:21:56.185366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 10:21:56.185366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 10:21:56.185366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:21:56.185366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 10:21:56.185366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:56.185366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 10:21:56.185366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:56.185366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 10:21:56.185366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 10:21:56.185366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 10:21:56.185366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 10:21:56.185366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 10:21:56.186652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 10:21:56.186652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 10:21:56.186652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 10:21:56.186652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 10:21:56.186652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 10:21:56.186652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 10:21:56.186652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 10:21:56.186652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 10:21:56.186652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 10:21:56.186652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 10:21:56.186652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 10:21:56.186652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 10:21:56.186652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 10:21:56.186652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 10:21:56.186652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 10:21:56.186652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 10:21:56.187599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 10:21:56.187599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application with Lock( Sep 19 10:21:56.187599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 10:21:56.187599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self._enter() Sep 19 10:21:56.187599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 10:21:56.187599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 10:21:56.187599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 10:21:56.187599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application return self.creator() Sep 19 10:21:56.187599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 10:21:56.187599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application created_value = creator( Sep 19 10:21:56.187599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 10:21:56.187599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 10:21:56.187599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 10:21:56.187599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: ERROR keystone.server.flask.application Sep 19 10:21:56.187599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 659/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Sep 19 10:21:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 10:21:56.192437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299a0a6-0c2d-4e38-9591-02d45676712b None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:56.266434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:21:56.288208 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299a0a6-0c2d-4e38-9591-02d45676712b None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:56.322474 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c95983-5f5a-4bc8-aa40-f1d5b3c4f87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:56.323002 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c95983-5f5a-4bc8-aa40-f1d5b3c4f87d None admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:56.323277 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c95983-5f5a-4bc8-aa40-f1d5b3c4f87d None admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:56.323548 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c95983-5f5a-4bc8-aa40-f1d5b3c4f87d None admin] PATH_INFO: `/v3/services` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:56.324169 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c95983-5f5a-4bc8-aa40-f1d5b3c4f87d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:21:56.324492 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c95983-5f5a-4bc8-aa40-f1d5b3c4f87d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:56.325882 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c95983-5f5a-4bc8-aa40-f1d5b3c4f87d None admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:56.330799 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 646/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Sep 19 10:21:56 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:21:56.336701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc50a3a-3008-4878-8d3e-54f6a48cb2b0 None admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:21:56.435068 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc50a3a-3008-4878-8d3e-54f6a48cb2b0 None admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:21:56.467570 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eea344-3d2a-43b7-8855-37755b926bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:21:56.468159 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eea344-3d2a-43b7-8855-37755b926bac None admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:21:56.468438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eea344-3d2a-43b7-8855-37755b926bac None admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:21:56.468708 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eea344-3d2a-43b7-8855-37755b926bac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:21:56.469403 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eea344-3d2a-43b7-8855-37755b926bac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:21:56.470797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eea344-3d2a-43b7-8855-37755b926bac None admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:21:56.534635 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 660/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:21:56 2022] POST /identity/v3/endpoints => generated 331 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:22:06.334272 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=30 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:22:06.530042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:27:36.228807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b9b14-6fbb-4b2b-b40d-6e0be48a6faf None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:36.229317 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b9b14-6fbb-4b2b-b40d-6e0be48a6faf None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:36.229473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b9b14-6fbb-4b2b-b40d-6e0be48a6faf None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:36.231082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 647/1307] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:27:36.237018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3df952-276d-4557-83e3-a9acdd5814de None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:36.237468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3df952-276d-4557-83e3-a9acdd5814de None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:36.237973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3df952-276d-4557-83e3-a9acdd5814de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:36.451194 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-2b3df952-276d-4557-83e3-a9acdd5814de None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:27:36.452845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2b3df952-276d-4557-83e3-a9acdd5814de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:36.587315 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 661/1308] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:27:36.886821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43860eba-1eb5-48fb-9f57-ea8ab5b42fb7 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:36.887475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43860eba-1eb5-48fb-9f57-ea8ab5b42fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:36.887982 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43860eba-1eb5-48fb-9f57-ea8ab5b42fb7 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:36.889972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 648/1309] 10.222.0.42 () {54 vars in 963 bytes} [Mon Sep 19 10:27:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:27:36.896995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d56fc-3071-4e22-bbe5-882136f7e573 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:36.897392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d56fc-3071-4e22-bbe5-882136f7e573 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:36.897576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d56fc-3071-4e22-bbe5-882136f7e573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:37.013641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-367d56fc-3071-4e22-bbe5-882136f7e573 None None] MFA Rules not processed for user `a282e74d177544bbbfba0fdb5e52df60`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:27:37.015302 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-367d56fc-3071-4e22-bbe5-882136f7e573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:37.041154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:27:37.119011 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43860eba-1eb5-48fb-9f57-ea8ab5b42fb7 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:27:37.120820 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-43860eba-1eb5-48fb-9f57-ea8ab5b42fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:37.125279 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 662/1310] 10.222.0.42 () {58 vars in 1060 bytes} [Mon Sep 19 10:27:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:27:37.239654 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43860eba-1eb5-48fb-9f57-ea8ab5b42fb7 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:27:37.320766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2031e51-7290-4b48-827c-2c5b0dc9be2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:27:37.321705 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2031e51-7290-4b48-827c-2c5b0dc9be2c service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:37.322203 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2031e51-7290-4b48-827c-2c5b0dc9be2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:37.322626 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2031e51-7290-4b48-827c-2c5b0dc9be2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:37.324413 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-a2031e51-7290-4b48-827c-2c5b0dc9be2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:37.387152 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:27:37.453828 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2031e51-7290-4b48-827c-2c5b0dc9be2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:27:37.455534 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2031e51-7290-4b48-827c-2c5b0dc9be2c service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:27:37.476018 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 649/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:27:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:27:39.938899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcacea7-75ce-4ba1-ad3c-c45b47fb55a9 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:39.939450 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcacea7-75ce-4ba1-ad3c-c45b47fb55a9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:39.939450 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcacea7-75ce-4ba1-ad3c-c45b47fb55a9 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:39.940769 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 663/1312] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:27:39.946528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cad104-2804-45f7-9fc8-f10e101eff54 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:39.946907 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cad104-2804-45f7-9fc8-f10e101eff54 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:39.947087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cad104-2804-45f7-9fc8-f10e101eff54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:40.016481 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-84cad104-2804-45f7-9fc8-f10e101eff54 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:27:40.017644 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-84cad104-2804-45f7-9fc8-f10e101eff54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:40.112663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 650/1313] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:27:40.404608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf67771-9726-436a-a7c7-b54744822c87 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:40.404889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf67771-9726-436a-a7c7-b54744822c87 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:40.405195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf67771-9726-436a-a7c7-b54744822c87 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:40.406537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 664/1314] 10.222.0.42 () {54 vars in 963 bytes} [Mon Sep 19 10:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:27:40.412398 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff80337-fe17-4291-8fc5-0d51c5067558 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:40.412685 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff80337-fe17-4291-8fc5-0d51c5067558 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:40.412960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff80337-fe17-4291-8fc5-0d51c5067558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:40.424952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:27:40.487995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-cff80337-fe17-4291-8fc5-0d51c5067558 None None] MFA Rules not processed for user `a282e74d177544bbbfba0fdb5e52df60`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:27:40.489224 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-cff80337-fe17-4291-8fc5-0d51c5067558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:40.562546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 651/1315] 10.222.0.42 () {58 vars in 1060 bytes} [Mon Sep 19 10:27:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:27:40.571585 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf67771-9726-436a-a7c7-b54744822c87 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:27:40.573174 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-edf67771-9726-436a-a7c7-b54744822c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:40.675366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf67771-9726-436a-a7c7-b54744822c87 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:27:40.744447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:27:40.746721 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f454e0-dd53-4301-8ebf-bfbfd2d4b204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:27:40.747320 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f454e0-dd53-4301-8ebf-bfbfd2d4b204 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:40.747603 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f454e0-dd53-4301-8ebf-bfbfd2d4b204 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:40.747879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f454e0-dd53-4301-8ebf-bfbfd2d4b204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:40.749499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-00f454e0-dd53-4301-8ebf-bfbfd2d4b204 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:40.856994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f454e0-dd53-4301-8ebf-bfbfd2d4b204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:27:40.858437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f454e0-dd53-4301-8ebf-bfbfd2d4b204 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:27:40.876133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 665/1316] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:27:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:27:43.747052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c40f3-1392-4f7a-b593-5b980d8c4001 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:43.747462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c40f3-1392-4f7a-b593-5b980d8c4001 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:43.747607 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c40f3-1392-4f7a-b593-5b980d8c4001 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:43.748952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 652/1317] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:27:43.754120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279792a-dde7-499b-a029-d1e313a86366 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:43.754551 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279792a-dde7-499b-a029-d1e313a86366 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:43.754672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279792a-dde7-499b-a029-d1e313a86366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:43.821047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f279792a-dde7-499b-a029-d1e313a86366 None None] MFA Rules not processed for user `fda9ca2d3fb54751b2eb0636906d0c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:27:43.822289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f279792a-dde7-499b-a029-d1e313a86366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:43.873901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:27:43.917999 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 666/1318] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:27:43 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:27:44.199996 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c40f3-1392-4f7a-b593-5b980d8c4001 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:27:44.201605 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-699c40f3-1392-4f7a-b593-5b980d8c4001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:44.307373 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c40f3-1392-4f7a-b593-5b980d8c4001 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:27:44.310198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:27:44.379502 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a443366-6f29-4787-b82b-72fddcf38292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:27:44.380108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a443366-6f29-4787-b82b-72fddcf38292 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:44.380383 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a443366-6f29-4787-b82b-72fddcf38292 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:44.380656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a443366-6f29-4787-b82b-72fddcf38292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:44.382284 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-0a443366-6f29-4787-b82b-72fddcf38292 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:44.488300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a443366-6f29-4787-b82b-72fddcf38292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:27:44.489685 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a443366-6f29-4787-b82b-72fddcf38292 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:27:44.508306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 653/1319] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:27:44 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:27:47.534288 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e225a82-bd0f-4e43-9e93-c8c8425ad918 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:47.534833 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e225a82-bd0f-4e43-9e93-c8c8425ad918 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:47.534833 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e225a82-bd0f-4e43-9e93-c8c8425ad918 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:47.536181 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 667/1320] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:27:47.541338 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e49654-9708-49ba-a867-8f9fa07a1f1e None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:47.541744 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e49654-9708-49ba-a867-8f9fa07a1f1e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:47.541894 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e49654-9708-49ba-a867-8f9fa07a1f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:47.583219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:27:47.608447 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-39e49654-9708-49ba-a867-8f9fa07a1f1e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:27:47.610336 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-39e49654-9708-49ba-a867-8f9fa07a1f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:47.703239 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 654/1321] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:27:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:27:47.997185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e225a82-bd0f-4e43-9e93-c8c8425ad918 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:27:47.998724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0e225a82-bd0f-4e43-9e93-c8c8425ad918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:48.099875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e225a82-bd0f-4e43-9e93-c8c8425ad918 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:27:48.172270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99448cdc-4424-4e0d-8a20-0e57e3636123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:27:48.172931 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99448cdc-4424-4e0d-8a20-0e57e3636123 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:48.173247 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99448cdc-4424-4e0d-8a20-0e57e3636123 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:48.173520 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99448cdc-4424-4e0d-8a20-0e57e3636123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:48.175115 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-99448cdc-4424-4e0d-8a20-0e57e3636123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:48.181902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:27:48.282226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99448cdc-4424-4e0d-8a20-0e57e3636123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:27:48.283634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99448cdc-4424-4e0d-8a20-0e57e3636123 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:27:48.301860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 668/1322] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:27:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:27:51.479760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff78b28-80e2-4447-8c72-6abe07f3debe None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:51.480291 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff78b28-80e2-4447-8c72-6abe07f3debe None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:51.480291 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff78b28-80e2-4447-8c72-6abe07f3debe None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:51.481659 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 655/1323] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:27:51.486921 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b929934-70fa-40f5-9f4f-30b0191274bf None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:51.487373 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b929934-70fa-40f5-9f4f-30b0191274bf None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:51.487561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b929934-70fa-40f5-9f4f-30b0191274bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:51.557787 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8b929934-70fa-40f5-9f4f-30b0191274bf None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:27:51.558997 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8b929934-70fa-40f5-9f4f-30b0191274bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:51.619223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:27:51.662307 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 669/1324] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:27:51.953356 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff78b28-80e2-4447-8c72-6abe07f3debe None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:27:51.954900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-6ff78b28-80e2-4447-8c72-6abe07f3debe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:52.065979 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff78b28-80e2-4447-8c72-6abe07f3debe None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:27:52.079485 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:27:52.140230 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ad797-a019-470a-a4e4-e71c566f63d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:27:52.140837 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ad797-a019-470a-a4e4-e71c566f63d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:52.141149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ad797-a019-470a-a4e4-e71c566f63d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:52.141423 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ad797-a019-470a-a4e4-e71c566f63d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:52.143015 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-484ad797-a019-470a-a4e4-e71c566f63d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:52.253077 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ad797-a019-470a-a4e4-e71c566f63d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:27:52.254530 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ad797-a019-470a-a4e4-e71c566f63d2 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:27:52.272415 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 656/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:27:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:27:54.486155 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8c873-d882-4bd5-854a-084171a92ae4 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:54.486739 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8c873-d882-4bd5-854a-084171a92ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:54.486739 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8c873-d882-4bd5-854a-084171a92ae4 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:54.488081 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 670/1326] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:27:54.494534 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8a91f-8dcb-451b-8597-90c555e7e125 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:54.495074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8a91f-8dcb-451b-8597-90c555e7e125 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:54.495074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8a91f-8dcb-451b-8597-90c555e7e125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:54.544177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:27:54.562603 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-f3e8a91f-8dcb-451b-8597-90c555e7e125 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:27:54.563811 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-f3e8a91f-8dcb-451b-8597-90c555e7e125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:54.659604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 657/1327] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:27:54.945051 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8c873-d882-4bd5-854a-084171a92ae4 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:27:54.946676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5cf8c873-d882-4bd5-854a-084171a92ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:55.051418 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8c873-d882-4bd5-854a-084171a92ae4 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:27:55.124844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34393c-5c5a-4a4c-a6ae-2c0722072310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:27:55.125475 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34393c-5c5a-4a4c-a6ae-2c0722072310 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:55.125750 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34393c-5c5a-4a4c-a6ae-2c0722072310 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:55.126018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34393c-5c5a-4a4c-a6ae-2c0722072310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:55.127588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4c34393c-5c5a-4a4c-a6ae-2c0722072310 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:55.145505 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:27:55.236072 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34393c-5c5a-4a4c-a6ae-2c0722072310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:27:55.237472 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34393c-5c5a-4a4c-a6ae-2c0722072310 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:27:55.256812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 671/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:27:57.496853 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384557c2-5ba9-4673-b6ec-63c32c69d488 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:57.497410 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384557c2-5ba9-4673-b6ec-63c32c69d488 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:57.497503 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384557c2-5ba9-4673-b6ec-63c32c69d488 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:57.498857 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 658/1329] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:27:57.504072 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d2f37-cddc-49f9-bcd3-b9fce91be595 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:57.504461 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d2f37-cddc-49f9-bcd3-b9fce91be595 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:57.504631 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d2f37-cddc-49f9-bcd3-b9fce91be595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:57.571455 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-984d2f37-cddc-49f9-bcd3-b9fce91be595 None None] MFA Rules not processed for user `fda9ca2d3fb54751b2eb0636906d0c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:27:57.572604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-984d2f37-cddc-49f9-bcd3-b9fce91be595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:57.753467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:27:57.789871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 672/1330] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:27:57 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:27:58.074301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384557c2-5ba9-4673-b6ec-63c32c69d488 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:27:58.075887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-384557c2-5ba9-4673-b6ec-63c32c69d488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:58.186270 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384557c2-5ba9-4673-b6ec-63c32c69d488 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:27:58.210751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:27:58.269385 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff1b67-5b1c-476c-83d4-df08ee91ae5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:27:58.270050 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff1b67-5b1c-476c-83d4-df08ee91ae5f service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:27:58.270332 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff1b67-5b1c-476c-83d4-df08ee91ae5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:27:58.270608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff1b67-5b1c-476c-83d4-df08ee91ae5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:27:58.272207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-67ff1b67-5b1c-476c-83d4-df08ee91ae5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:27:58.381032 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff1b67-5b1c-476c-83d4-df08ee91ae5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:27:58.383034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff1b67-5b1c-476c-83d4-df08ee91ae5f service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:27:58.401388 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 659/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:27:58 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:28:01.164559 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798efe4-5876-45c7-96df-2b19107cdf75 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:01.165212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798efe4-5876-45c7-96df-2b19107cdf75 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:01.165212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798efe4-5876-45c7-96df-2b19107cdf75 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:01.166482 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 673/1332] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:28:01.171847 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1734f78-c119-4a26-a9ef-ff6318a8052a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:01.173483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1734f78-c119-4a26-a9ef-ff6318a8052a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:01.173483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1734f78-c119-4a26-a9ef-ff6318a8052a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:01.235132 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:01.237796 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-e1734f78-c119-4a26-a9ef-ff6318a8052a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:01.238998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e1734f78-c119-4a26-a9ef-ff6318a8052a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:01.335779 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 660/1333] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:28:01.618286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4798efe4-5876-45c7-96df-2b19107cdf75 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:28:01.619858 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4798efe4-5876-45c7-96df-2b19107cdf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:01.720518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4798efe4-5876-45c7-96df-2b19107cdf75 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:28:01.793297 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e47555-cdcd-414e-a9b1-e7e7f3d9dc83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:28:01.793892 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e47555-cdcd-414e-a9b1-e7e7f3d9dc83 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:01.794167 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e47555-cdcd-414e-a9b1-e7e7f3d9dc83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:01.794438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e47555-cdcd-414e-a9b1-e7e7f3d9dc83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:01.796006 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-89e47555-cdcd-414e-a9b1-e7e7f3d9dc83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:01.824143 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:01.904037 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e47555-cdcd-414e-a9b1-e7e7f3d9dc83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:28:01.905431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e47555-cdcd-414e-a9b1-e7e7f3d9dc83 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:28:01.924116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 674/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:28:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:28:04.097642 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb5357-2d8e-40b9-8d51-07f274f3c025 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:04.098208 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb5357-2d8e-40b9-8d51-07f274f3c025 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:04.098208 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb5357-2d8e-40b9-8d51-07f274f3c025 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:04.099446 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 661/1335] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:28:04.104925 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19da634-afad-4d60-af4b-e934a8df7d01 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:04.105347 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19da634-afad-4d60-af4b-e934a8df7d01 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:04.105510 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19da634-afad-4d60-af4b-e934a8df7d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:04.171738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e19da634-afad-4d60-af4b-e934a8df7d01 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:04.173036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e19da634-afad-4d60-af4b-e934a8df7d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:04.246915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:04.268309 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 675/1336] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:28:04.555267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb5357-2d8e-40b9-8d51-07f274f3c025 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:28:04.556825 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-0eeb5357-2d8e-40b9-8d51-07f274f3c025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:04.657914 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb5357-2d8e-40b9-8d51-07f274f3c025 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:28:04.685684 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:04.729386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e8156-ef13-41e0-89a6-4521daa12ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:28:04.729986 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e8156-ef13-41e0-89a6-4521daa12ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:04.730261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e8156-ef13-41e0-89a6-4521daa12ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:04.730532 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e8156-ef13-41e0-89a6-4521daa12ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:04.732100 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c54e8156-ef13-41e0-89a6-4521daa12ed3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:04.838748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e8156-ef13-41e0-89a6-4521daa12ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:28:04.840191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e8156-ef13-41e0-89a6-4521daa12ed3 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:28:04.858414 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 662/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:28:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:28:07.587800 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da2e4e-c60f-45a5-aa5d-54376a2eb8ec None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:07.588455 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da2e4e-c60f-45a5-aa5d-54376a2eb8ec None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:07.588588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da2e4e-c60f-45a5-aa5d-54376a2eb8ec None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:07.589889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 676/1338] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:28:07.594892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb70c5-091d-4e74-af87-a2df9f0c650d None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:07.595335 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb70c5-091d-4e74-af87-a2df9f0c650d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:07.595442 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb70c5-091d-4e74-af87-a2df9f0c650d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:07.660446 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-dddb70c5-091d-4e74-af87-a2df9f0c650d None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:07.661668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-dddb70c5-091d-4e74-af87-a2df9f0c650d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:07.669789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:07.758106 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 663/1339] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:28:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:28:08.041107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da2e4e-c60f-45a5-aa5d-54376a2eb8ec None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:28:08.042647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-20da2e4e-c60f-45a5-aa5d-54376a2eb8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:08.144339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da2e4e-c60f-45a5-aa5d-54376a2eb8ec None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:28:08.220811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e0c91-e36d-4bb8-8e67-ff762abb187a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:28:08.221463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e0c91-e36d-4bb8-8e67-ff762abb187a service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:08.221719 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e0c91-e36d-4bb8-8e67-ff762abb187a service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:08.221981 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e0c91-e36d-4bb8-8e67-ff762abb187a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:08.223549 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5d8e0c91-e36d-4bb8-8e67-ff762abb187a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:08.261894 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:08.333140 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e0c91-e36d-4bb8-8e67-ff762abb187a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:28:08.334540 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e0c91-e36d-4bb8-8e67-ff762abb187a service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:28:08.354526 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 677/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:28:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:28:12.709281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf3ee3-4cb3-437a-86e3-753c97aec2d3 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:12.710317 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf3ee3-4cb3-437a-86e3-753c97aec2d3 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:12.710399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf3ee3-4cb3-437a-86e3-753c97aec2d3 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:12.711806 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 664/1341] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:28:12.717048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0c548-7c13-4127-998d-e673351a21e2 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:12.717502 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0c548-7c13-4127-998d-e673351a21e2 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:12.717692 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0c548-7c13-4127-998d-e673351a21e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:12.795362 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-aba0c548-7c13-4127-998d-e673351a21e2 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:12.796557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-aba0c548-7c13-4127-998d-e673351a21e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:12.883308 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:12.894260 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 678/1342] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:28:13.179646 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf3ee3-4cb3-437a-86e3-753c97aec2d3 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:28:13.181205 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-a4cf3ee3-4cb3-437a-86e3-753c97aec2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:13.285159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf3ee3-4cb3-437a-86e3-753c97aec2d3 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:28:13.323348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:13.359045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff3616-82c7-4a99-82ad-fead12f33552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:28:13.359649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff3616-82c7-4a99-82ad-fead12f33552 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:13.359925 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff3616-82c7-4a99-82ad-fead12f33552 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:13.360194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff3616-82c7-4a99-82ad-fead12f33552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:13.361812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-acff3616-82c7-4a99-82ad-fead12f33552 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:13.471012 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff3616-82c7-4a99-82ad-fead12f33552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:28:13.472495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff3616-82c7-4a99-82ad-fead12f33552 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:28:13.491764 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 665/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:28:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:28:16.624282 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c77bfb-a803-448b-8f48-0e20e8dcc160 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:16.624826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c77bfb-a803-448b-8f48-0e20e8dcc160 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:16.624826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c77bfb-a803-448b-8f48-0e20e8dcc160 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:16.626212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 679/1344] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:28:16.632216 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44333c8-f3be-4b69-b1d9-eac0b8fd1800 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:16.632629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44333c8-f3be-4b69-b1d9-eac0b8fd1800 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:16.632743 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44333c8-f3be-4b69-b1d9-eac0b8fd1800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:16.699801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-d44333c8-f3be-4b69-b1d9-eac0b8fd1800 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:16.701072 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d44333c8-f3be-4b69-b1d9-eac0b8fd1800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:16.722505 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:16.802037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 666/1345] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:28:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:28:17.085967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c77bfb-a803-448b-8f48-0e20e8dcc160 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:28:17.087493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b5c77bfb-a803-448b-8f48-0e20e8dcc160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:17.192359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c77bfb-a803-448b-8f48-0e20e8dcc160 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:28:17.267874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cf1e6-2631-4f07-b8a3-7b6b4d68522f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:28:17.268485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cf1e6-2631-4f07-b8a3-7b6b4d68522f service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:17.268765 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cf1e6-2631-4f07-b8a3-7b6b4d68522f service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:17.270157 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cf1e6-2631-4f07-b8a3-7b6b4d68522f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:17.270739 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-859cf1e6-2631-4f07-b8a3-7b6b4d68522f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:17.315194 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:17.382046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859cf1e6-2631-4f07-b8a3-7b6b4d68522f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:28:17.383473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859cf1e6-2631-4f07-b8a3-7b6b4d68522f service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:28:17.422967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 680/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:28:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:28:22.011644 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff9b93-dbf9-45d2-9498-0dde1a1b8a81 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:22.012171 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff9b93-dbf9-45d2-9498-0dde1a1b8a81 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:22.012171 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff9b93-dbf9-45d2-9498-0dde1a1b8a81 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:22.016682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 667/1347] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:28:22.019092 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a13ae-a124-47b3-9627-4e54f081b9ce None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:22.019524 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a13ae-a124-47b3-9627-4e54f081b9ce None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:22.019683 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a13ae-a124-47b3-9627-4e54f081b9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:22.091449 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6c7a13ae-a124-47b3-9627-4e54f081b9ce None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:22.092746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6c7a13ae-a124-47b3-9627-4e54f081b9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:22.194506 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:22.196669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 681/1348] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:28:22.478907 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff9b93-dbf9-45d2-9498-0dde1a1b8a81 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:28:22.480442 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e4ff9b93-dbf9-45d2-9498-0dde1a1b8a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:22.581179 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff9b93-dbf9-45d2-9498-0dde1a1b8a81 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:28:22.631073 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:22.656618 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07695efb-ac6a-40d5-8bb1-9085ab45b45f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:28:22.657268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07695efb-ac6a-40d5-8bb1-9085ab45b45f service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:22.657537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07695efb-ac6a-40d5-8bb1-9085ab45b45f service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:22.657811 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07695efb-ac6a-40d5-8bb1-9085ab45b45f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:22.659403 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-07695efb-ac6a-40d5-8bb1-9085ab45b45f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:22.766680 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07695efb-ac6a-40d5-8bb1-9085ab45b45f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:28:22.768027 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07695efb-ac6a-40d5-8bb1-9085ab45b45f service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:28:22.786114 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 668/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:28:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:28:27.698242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ba78c-ea17-439c-af01-29f07c001d36 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:27.698706 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ba78c-ea17-439c-af01-29f07c001d36 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:27.699093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ba78c-ea17-439c-af01-29f07c001d36 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:27.700849 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 682/1350] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 19 10:28:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:28:27.715636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9865d0-72d5-4704-95ca-fd060f74c136 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:27.715636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9865d0-72d5-4704-95ca-fd060f74c136 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:27.715636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9865d0-72d5-4704-95ca-fd060f74c136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:27.863439 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-1d9865d0-72d5-4704-95ca-fd060f74c136 None None] MFA Rules not processed for user `ce0ef47a60f54ca1853fd267e308c58b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:27.865152 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-1d9865d0-72d5-4704-95ca-fd060f74c136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:27.885935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:27.994940 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 669/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 19 10:28:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:28:32.180259 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520477f-990e-47be-b168-7b62fc0ec1ad None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:32.180827 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520477f-990e-47be-b168-7b62fc0ec1ad None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:32.180827 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520477f-990e-47be-b168-7b62fc0ec1ad None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:32.182430 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 683/1352] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 19 10:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:28:32.187388 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1f64a-a1b4-4e96-ad40-dfedf20a953b None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:32.187792 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1f64a-a1b4-4e96-ad40-dfedf20a953b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:32.187944 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1f64a-a1b4-4e96-ad40-dfedf20a953b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:32.268469 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-63a1f64a-a1b4-4e96-ad40-dfedf20a953b None None] MFA Rules not processed for user `ce0ef47a60f54ca1853fd267e308c58b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:32.269696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-63a1f64a-a1b4-4e96-ad40-dfedf20a953b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:32.342334 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:32.386595 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 670/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 19 10:28:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:28:33.629235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a818ff-ace2-40b6-95d9-692b53e2df6a None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:33.629765 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a818ff-ace2-40b6-95d9-692b53e2df6a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:33.629765 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a818ff-ace2-40b6-95d9-692b53e2df6a None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:33.631620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 684/1354] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 19 10:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:28:33.636751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36e994-2923-4b68-a8dc-6597ba85d98a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:33.637233 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36e994-2923-4b68-a8dc-6597ba85d98a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:33.637332 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36e994-2923-4b68-a8dc-6597ba85d98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:33.714872 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-9c36e994-2923-4b68-a8dc-6597ba85d98a None None] MFA Rules not processed for user `ce0ef47a60f54ca1853fd267e308c58b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:33.716087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-9c36e994-2923-4b68-a8dc-6597ba85d98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:33.832393 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 671/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 19 10:28:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:28:37.456831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf004b-29e9-4f31-a005-ee5edd85d24d None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:37.457463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf004b-29e9-4f31-a005-ee5edd85d24d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:37.457463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf004b-29e9-4f31-a005-ee5edd85d24d None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:37.458996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 685/1356] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:28:37.464464 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e279aa-b784-4c17-91b9-6f14848a2677 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:37.464983 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e279aa-b784-4c17-91b9-6f14848a2677 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:37.465114 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e279aa-b784-4c17-91b9-6f14848a2677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:37.503011 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:37.536476 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-56e279aa-b784-4c17-91b9-6f14848a2677 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:37.538445 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-56e279aa-b784-4c17-91b9-6f14848a2677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:37.641684 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 672/1357] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:28:37.949358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b873c96-fb0b-4681-b446-b96682cca308 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:37.950388 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b873c96-fb0b-4681-b446-b96682cca308 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:37.950961 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b873c96-fb0b-4681-b446-b96682cca308 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:37.953099 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 686/1358] 10.222.0.42 () {54 vars in 965 bytes} [Mon Sep 19 10:28:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:28:37.959880 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d105b-14e6-48cd-aaaf-9d34b265a468 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:37.960426 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d105b-14e6-48cd-aaaf-9d34b265a468 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:37.961142 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d105b-14e6-48cd-aaaf-9d34b265a468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:38.080200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-1b0d105b-14e6-48cd-aaaf-9d34b265a468 None None] MFA Rules not processed for user `b4007794a30247c4a318af62d05e3b92`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:38.083811 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-1b0d105b-14e6-48cd-aaaf-9d34b265a468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:38.148676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:38.245954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 673/1359] 10.222.0.42 () {58 vars in 1062 bytes} [Mon Sep 19 10:28:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:28:38.246627 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b873c96-fb0b-4681-b446-b96682cca308 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:28:38.248153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3b873c96-fb0b-4681-b446-b96682cca308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:38.414355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b873c96-fb0b-4681-b446-b96682cca308 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:28:38.493712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:38.536951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707657d-da18-4adf-8953-a63f5a75071f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:28:38.537640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707657d-da18-4adf-8953-a63f5a75071f service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:38.537922 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707657d-da18-4adf-8953-a63f5a75071f service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:38.538199 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707657d-da18-4adf-8953-a63f5a75071f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:38.539815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3707657d-da18-4adf-8953-a63f5a75071f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:38.642388 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3707657d-da18-4adf-8953-a63f5a75071f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:28:38.643926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3707657d-da18-4adf-8953-a63f5a75071f service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:28:38.663788 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 687/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:28:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:28:41.048088 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4f273-27b9-4355-9356-4b2d50966723 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:41.048741 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4f273-27b9-4355-9356-4b2d50966723 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:41.048741 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4f273-27b9-4355-9356-4b2d50966723 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:41.053955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 674/1361] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:28:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:28:41.055208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0f63c-e5f8-4707-a5f5-51cd3abc72be None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:41.055674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0f63c-e5f8-4707-a5f5-51cd3abc72be None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:41.055674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0f63c-e5f8-4707-a5f5-51cd3abc72be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:41.132867 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a2a0f63c-e5f8-4707-a5f5-51cd3abc72be None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:41.134889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a2a0f63c-e5f8-4707-a5f5-51cd3abc72be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:41.179126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:41.243758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 688/1362] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:28:41.533292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4f273-27b9-4355-9356-4b2d50966723 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:28:41.534826 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-16f4f273-27b9-4355-9356-4b2d50966723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:41.708890 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4f273-27b9-4355-9356-4b2d50966723 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:28:41.719337 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:41.844997 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7a5ae-1b9e-4427-868c-8315d25a1ea7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:28:41.845638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7a5ae-1b9e-4427-868c-8315d25a1ea7 service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:41.845910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7a5ae-1b9e-4427-868c-8315d25a1ea7 service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:41.846173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7a5ae-1b9e-4427-868c-8315d25a1ea7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:41.847751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b9c7a5ae-1b9e-4427-868c-8315d25a1ea7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:41.942609 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7a5ae-1b9e-4427-868c-8315d25a1ea7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:28:41.944137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7a5ae-1b9e-4427-868c-8315d25a1ea7 service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:28:41.962986 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 675/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:28:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:28:44.572691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467e0be-4c28-4737-a8c1-16689306b2b1 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:44.573309 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467e0be-4c28-4737-a8c1-16689306b2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:44.573309 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467e0be-4c28-4737-a8c1-16689306b2b1 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:44.574584 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 689/1364] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:28:44.580050 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd583646-7538-41d7-b736-749d90ac908e None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:44.580440 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd583646-7538-41d7-b736-749d90ac908e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:44.580608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd583646-7538-41d7-b736-749d90ac908e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:44.596478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:44.648146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-fd583646-7538-41d7-b736-749d90ac908e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:44.649404 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-fd583646-7538-41d7-b736-749d90ac908e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:44.758177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 676/1365] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:28:45.057585 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0fd72-6ee6-4a2f-b8c2-f2ba528138df None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:45.058098 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0fd72-6ee6-4a2f-b8c2-f2ba528138df None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:45.058482 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0fd72-6ee6-4a2f-b8c2-f2ba528138df None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:45.060215 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 690/1366] 10.222.0.42 () {54 vars in 965 bytes} [Mon Sep 19 10:28:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:28:45.065693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d58d8-5203-4fc7-9aad-ff001cda5ef9 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:45.065985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d58d8-5203-4fc7-9aad-ff001cda5ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:45.066273 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d58d8-5203-4fc7-9aad-ff001cda5ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:45.146886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-9c1d58d8-5203-4fc7-9aad-ff001cda5ef9 None None] MFA Rules not processed for user `b4007794a30247c4a318af62d05e3b92`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:45.148037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-9c1d58d8-5203-4fc7-9aad-ff001cda5ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:45.187453 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:45.277641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 677/1367] 10.222.0.42 () {58 vars in 1062 bytes} [Mon Sep 19 10:28:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:28:45.285781 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe0fd72-6ee6-4a2f-b8c2-f2ba528138df None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:28:45.287560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bfe0fd72-6ee6-4a2f-b8c2-f2ba528138df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:45.446014 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe0fd72-6ee6-4a2f-b8c2-f2ba528138df None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:28:45.467531 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:45.566630 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1488559-a70c-4340-8b24-7c4fb9abd6d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:28:45.567405 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1488559-a70c-4340-8b24-7c4fb9abd6d9 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:45.567815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1488559-a70c-4340-8b24-7c4fb9abd6d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:45.568188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1488559-a70c-4340-8b24-7c4fb9abd6d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:45.569954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f1488559-a70c-4340-8b24-7c4fb9abd6d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:45.670926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1488559-a70c-4340-8b24-7c4fb9abd6d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:28:45.672684 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1488559-a70c-4340-8b24-7c4fb9abd6d9 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:28:45.692525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 691/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:28:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:28:45.744077 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d468e-a2e8-49cf-9587-b8e952934cea None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:45.744659 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d468e-a2e8-49cf-9587-b8e952934cea None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:45.744659 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d468e-a2e8-49cf-9587-b8e952934cea None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:45.746326 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 678/1369] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:28:45.752125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51a6df-2e10-4f3c-8d12-e8a074c294cc None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:45.752434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51a6df-2e10-4f3c-8d12-e8a074c294cc None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:45.752748 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51a6df-2e10-4f3c-8d12-e8a074c294cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:45.791616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:45.834136 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7f51a6df-2e10-4f3c-8d12-e8a074c294cc None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:45.835758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7f51a6df-2e10-4f3c-8d12-e8a074c294cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:45.944041 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 692/1370] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:28:46.280328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6876c83-50f3-444d-9dfe-083179eb2874 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:46.280328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6876c83-50f3-444d-9dfe-083179eb2874 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:46.280328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6876c83-50f3-444d-9dfe-083179eb2874 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:46.281632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 679/1371] 10.222.0.42 () {54 vars in 962 bytes} [Mon Sep 19 10:28:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:28:46.286654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc94a5c-93bb-468c-9215-01b8b56ea35e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:46.287017 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc94a5c-93bb-468c-9215-01b8b56ea35e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:46.287361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc94a5c-93bb-468c-9215-01b8b56ea35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:46.370839 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1bc94a5c-93bb-468c-9215-01b8b56ea35e None None] MFA Rules not processed for user `e8a10f3332584a90a2231564527078cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:46.372450 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1bc94a5c-93bb-468c-9215-01b8b56ea35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:46.422465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:46.453107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 693/1372] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Sep 19 10:28:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:28:46.460678 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6876c83-50f3-444d-9dfe-083179eb2874 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:28:46.462263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b6876c83-50f3-444d-9dfe-083179eb2874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:46.575968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6876c83-50f3-444d-9dfe-083179eb2874 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:28:46.616835 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:46.650828 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b65e4-4e10-4042-bc0d-9c2e66b00c54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:28:46.651435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b65e4-4e10-4042-bc0d-9c2e66b00c54 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:46.651713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b65e4-4e10-4042-bc0d-9c2e66b00c54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:46.652237 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b65e4-4e10-4042-bc0d-9c2e66b00c54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:46.653872 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-836b65e4-4e10-4042-bc0d-9c2e66b00c54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:46.785189 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836b65e4-4e10-4042-bc0d-9c2e66b00c54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:28:46.787228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836b65e4-4e10-4042-bc0d-9c2e66b00c54 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:28:46.807796 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 680/1373] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:28:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:28:48.123926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697dc51-2ebb-44f6-9875-8fbc956cfd2f None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:48.124442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697dc51-2ebb-44f6-9875-8fbc956cfd2f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:48.124442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697dc51-2ebb-44f6-9875-8fbc956cfd2f None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:48.125778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 694/1374] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:28:48.131866 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55cfff-05d6-48c7-a46b-9c9247489462 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:48.132299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55cfff-05d6-48c7-a46b-9c9247489462 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:48.132621 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55cfff-05d6-48c7-a46b-9c9247489462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:48.201310 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-1f55cfff-05d6-48c7-a46b-9c9247489462 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:48.202548 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-1f55cfff-05d6-48c7-a46b-9c9247489462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:48.224005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:48.302790 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 681/1375] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:28:48.593751 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697dc51-2ebb-44f6-9875-8fbc956cfd2f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:28:48.595276 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b697dc51-2ebb-44f6-9875-8fbc956cfd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:48.743205 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697dc51-2ebb-44f6-9875-8fbc956cfd2f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:28:48.799116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:48.869290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920295c6-3cb8-4741-ad3f-aeb83814a8c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:28:48.869910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920295c6-3cb8-4741-ad3f-aeb83814a8c2 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:48.870190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920295c6-3cb8-4741-ad3f-aeb83814a8c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:48.870465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920295c6-3cb8-4741-ad3f-aeb83814a8c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:48.872059 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-920295c6-3cb8-4741-ad3f-aeb83814a8c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:48.965240 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920295c6-3cb8-4741-ad3f-aeb83814a8c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:28:48.966879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920295c6-3cb8-4741-ad3f-aeb83814a8c2 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:28:48.986679 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 695/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:28:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:28:51.339685 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c41965-4be6-4445-b0dd-7ec7ac7000a1 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:51.340543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c41965-4be6-4445-b0dd-7ec7ac7000a1 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:51.340975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c41965-4be6-4445-b0dd-7ec7ac7000a1 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:51.342481 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 682/1377] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:28:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:28:51.347683 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52146627-7c0a-4a2f-a47d-daccf68d4199 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:51.348090 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52146627-7c0a-4a2f-a47d-daccf68d4199 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:51.348289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52146627-7c0a-4a2f-a47d-daccf68d4199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:51.418025 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-52146627-7c0a-4a2f-a47d-daccf68d4199 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:51.419244 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-52146627-7c0a-4a2f-a47d-daccf68d4199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:51.427344 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:51.524669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 696/1378] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:28:51.816691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c41965-4be6-4445-b0dd-7ec7ac7000a1 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:28:51.818395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-22c41965-4be6-4445-b0dd-7ec7ac7000a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:51.977531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c41965-4be6-4445-b0dd-7ec7ac7000a1 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:28:51.981679 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:52.098472 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a77f0b-def3-48e8-a03a-4b5a4c0aa058 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:28:52.099258 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a77f0b-def3-48e8-a03a-4b5a4c0aa058 service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:52.099636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a77f0b-def3-48e8-a03a-4b5a4c0aa058 service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:52.100009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a77f0b-def3-48e8-a03a-4b5a4c0aa058 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:52.101765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-04a77f0b-def3-48e8-a03a-4b5a4c0aa058 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:52.198151 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a77f0b-def3-48e8-a03a-4b5a4c0aa058 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:28:52.199906 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a77f0b-def3-48e8-a03a-4b5a4c0aa058 service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:28:52.220661 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 683/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:28:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:28:54.654439 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad105bbe-e259-4c6d-abba-487d82597efe None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:54.655094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad105bbe-e259-4c6d-abba-487d82597efe None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:54.655094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad105bbe-e259-4c6d-abba-487d82597efe None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:54.656323 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 697/1380] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:28:54.662309 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0a511-d5fa-4fa0-afe4-2d4a06b7b34d None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:54.662767 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0a511-d5fa-4fa0-afe4-2d4a06b7b34d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:54.662887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0a511-d5fa-4fa0-afe4-2d4a06b7b34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:54.674255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:54.742005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-5fb0a511-d5fa-4fa0-afe4-2d4a06b7b34d None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:54.743408 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5fb0a511-d5fa-4fa0-afe4-2d4a06b7b34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:54.842262 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 684/1381] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:28:55.130508 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad105bbe-e259-4c6d-abba-487d82597efe None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:28:55.132031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ad105bbe-e259-4c6d-abba-487d82597efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:55.274392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:55.285030 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad105bbe-e259-4c6d-abba-487d82597efe None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:28:55.408176 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c5ed8-7550-45a8-923d-15d1f512f323 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:28:55.408784 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c5ed8-7550-45a8-923d-15d1f512f323 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:55.409076 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c5ed8-7550-45a8-923d-15d1f512f323 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:55.409359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c5ed8-7550-45a8-923d-15d1f512f323 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:55.410963 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1c0c5ed8-7550-45a8-923d-15d1f512f323 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:55.500504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c5ed8-7550-45a8-923d-15d1f512f323 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:28:55.502761 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c5ed8-7550-45a8-923d-15d1f512f323 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:28:55.515334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:55.525882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 698/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:28:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:28:57.850836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d79337-7b31-430a-a368-cae62fcba5e4 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:57.851339 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d79337-7b31-430a-a368-cae62fcba5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:57.851434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d79337-7b31-430a-a368-cae62fcba5e4 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:57.852784 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 685/1383] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:28:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:28:57.858400 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2af88-bebb-4d8c-ba2c-aac0f9d0fec8 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:57.858824 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2af88-bebb-4d8c-ba2c-aac0f9d0fec8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:57.859038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2af88-bebb-4d8c-ba2c-aac0f9d0fec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:57.925313 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-bdf2af88-bebb-4d8c-ba2c-aac0f9d0fec8 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:28:57.927031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bdf2af88-bebb-4d8c-ba2c-aac0f9d0fec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:58.030208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 699/1384] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:28:58.330015 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d79337-7b31-430a-a368-cae62fcba5e4 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:28:58.331650 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-95d79337-7b31-430a-a368-cae62fcba5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:58.436079 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:58.498181 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d79337-7b31-430a-a368-cae62fcba5e4 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:28:58.612847 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e1b39-a855-4fe8-a9e0-7b6fc47ecba6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:28:58.613400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e1b39-a855-4fe8-a9e0-7b6fc47ecba6 service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:28:58.613736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e1b39-a855-4fe8-a9e0-7b6fc47ecba6 service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:28:58.613944 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e1b39-a855-4fe8-a9e0-7b6fc47ecba6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:28:58.615544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-290e1b39-a855-4fe8-a9e0-7b6fc47ecba6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:28:58.667339 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:28:58.708660 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e1b39-a855-4fe8-a9e0-7b6fc47ecba6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:28:58.710393 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e1b39-a855-4fe8-a9e0-7b6fc47ecba6 service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:28:58.729188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 686/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:28:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:29:00.957535 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec69514-0af8-485d-a6d3-65c294726e87 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:00.958134 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec69514-0af8-485d-a6d3-65c294726e87 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:00.958134 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec69514-0af8-485d-a6d3-65c294726e87 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:00.959596 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 700/1386] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:00.964994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883dc34-e2ac-48b1-9624-5046eb30f4f1 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:00.965474 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883dc34-e2ac-48b1-9624-5046eb30f4f1 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:00.965614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883dc34-e2ac-48b1-9624-5046eb30f4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:01.033891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-e883dc34-e2ac-48b1-9624-5046eb30f4f1 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:01.035091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e883dc34-e2ac-48b1-9624-5046eb30f4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:01.130308 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 687/1387] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:29:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:01.420444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec69514-0af8-485d-a6d3-65c294726e87 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:01.422054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4ec69514-0af8-485d-a6d3-65c294726e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:01.498118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:01.566480 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec69514-0af8-485d-a6d3-65c294726e87 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:01.679128 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ded98-a415-4a6f-ad7e-60da4145da61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:01.679640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ded98-a415-4a6f-ad7e-60da4145da61 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:01.679912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ded98-a415-4a6f-ad7e-60da4145da61 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:01.680184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ded98-a415-4a6f-ad7e-60da4145da61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:01.681767 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2d1ded98-a415-4a6f-ad7e-60da4145da61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:01.720794 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:01.778851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ded98-a415-4a6f-ad7e-60da4145da61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:01.780452 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ded98-a415-4a6f-ad7e-60da4145da61 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:01.799204 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 701/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:29:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:29:02.285017 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83ea65-bf24-42ea-8bae-b6f606bb893c None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:02.285599 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83ea65-bf24-42ea-8bae-b6f606bb893c None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:02.285599 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83ea65-bf24-42ea-8bae-b6f606bb893c None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:02.286915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 688/1389] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:02.292202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5d98d-6caf-4afe-8728-bd786c6d9d41 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:02.292651 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5d98d-6caf-4afe-8728-bd786c6d9d41 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:02.292746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5d98d-6caf-4afe-8728-bd786c6d9d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:02.358687 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b8e5d98d-6caf-4afe-8728-bd786c6d9d41 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:02.359882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b8e5d98d-6caf-4afe-8728-bd786c6d9d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:02.458742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 702/1390] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:29:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:02.762770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9d798-5d13-4dd5-968d-5c1275a7a577 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:02.763121 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9d798-5d13-4dd5-968d-5c1275a7a577 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:02.763470 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9d798-5d13-4dd5-968d-5c1275a7a577 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:02.764898 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 689/1391] 10.222.0.42 () {54 vars in 962 bytes} [Mon Sep 19 10:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:02.770874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c160e-a230-4483-b305-767b60a51870 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:02.771239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c160e-a230-4483-b305-767b60a51870 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:02.771597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c160e-a230-4483-b305-767b60a51870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:02.789675 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:02.860391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8a0c160e-a230-4483-b305-767b60a51870 None None] MFA Rules not processed for user `a40b1ed93a6b4e5fb0e8f907627379a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:02.861741 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8a0c160e-a230-4483-b305-767b60a51870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:02.938335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 703/1392] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Sep 19 10:29:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:02.946215 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9d798-5d13-4dd5-968d-5c1275a7a577 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:02.947788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-52f9d798-5d13-4dd5-968d-5c1275a7a577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:02.979391 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:03.052468 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9d798-5d13-4dd5-968d-5c1275a7a577 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:03.128676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad85e6-2ada-406b-ae04-ca423bf02f15 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a40b1ed93a6b4e5fb0e8f907627379a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:03.129449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad85e6-2ada-406b-ae04-ca423bf02f15 service glance] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:03.129729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad85e6-2ada-406b-ae04-ca423bf02f15 service glance] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:03.130005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad85e6-2ada-406b-ae04-ca423bf02f15 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:03.131637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-19ad85e6-2ada-406b-ae04-ca423bf02f15 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:03.211663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:03.243796 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad85e6-2ada-406b-ae04-ca423bf02f15 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:03.245264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad85e6-2ada-406b-ae04-ca423bf02f15 service glance] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:03.263642 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 690/1393] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:29:04.093322 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae89e3-14bb-460f-9d4d-aeb55d801a31 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:04.093885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae89e3-14bb-460f-9d4d-aeb55d801a31 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:04.093885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae89e3-14bb-460f-9d4d-aeb55d801a31 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:04.095273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 704/1394] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:04.101311 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efd89f-73bf-4a55-b67e-2beab6fc6680 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:04.101746 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efd89f-73bf-4a55-b67e-2beab6fc6680 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:04.101862 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efd89f-73bf-4a55-b67e-2beab6fc6680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:04.170462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-96efd89f-73bf-4a55-b67e-2beab6fc6680 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:04.171705 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-96efd89f-73bf-4a55-b67e-2beab6fc6680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:04.187609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232d42e-7399-4efe-baec-b87e88d30b85 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:04.187891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232d42e-7399-4efe-baec-b87e88d30b85 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:04.188189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232d42e-7399-4efe-baec-b87e88d30b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:04.259066 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7232d42e-7399-4efe-baec-b87e88d30b85 None None] MFA Rules not processed for user `4ae237601b4a4aab9e969fedb5ee37d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:04.260280 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7232d42e-7399-4efe-baec-b87e88d30b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:04.272939 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 691/1395] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:04.318238 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:04.349637 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 705/1396] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 19 10:29:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:04.396028 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a61f9-33f5-4f79-980e-bca859c3580f None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:04.396316 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a61f9-33f5-4f79-980e-bca859c3580f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:04.396590 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a61f9-33f5-4f79-980e-bca859c3580f None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:04.398005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 692/1397] 10.222.0.42 () {54 vars in 943 bytes} [Mon Sep 19 10:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:04.404676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6582b2a-418e-40ab-b0ed-d2dd535de75b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:04.405024 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6582b2a-418e-40ab-b0ed-d2dd535de75b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:04.405347 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6582b2a-418e-40ab-b0ed-d2dd535de75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:04.481761 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-d6582b2a-418e-40ab-b0ed-d2dd535de75b None None] MFA Rules not processed for user `17a8bc6c506c45be897bda23eaab485a`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:04.482967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d6582b2a-418e-40ab-b0ed-d2dd535de75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:04.566664 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a61f9-33f5-4f79-980e-bca859c3580f None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:04.568491 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-917a61f9-33f5-4f79-980e-bca859c3580f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:04.589325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 706/1398] 10.222.0.42 () {58 vars in 1040 bytes} [Mon Sep 19 10:29:04 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:04.599383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6582b2a-418e-40ab-b0ed-d2dd535de75b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:04.609986 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6582b2a-418e-40ab-b0ed-d2dd535de75b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:04.618525 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:04.635252 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:04.716588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01948da2-01a3-498a-859c-6b75e328b57d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17a8bc6c506c45be897bda23eaab485a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:04.717232 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01948da2-01a3-498a-859c-6b75e328b57d service swift] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:04.717507 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01948da2-01a3-498a-859c-6b75e328b57d service swift] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:04.717776 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01948da2-01a3-498a-859c-6b75e328b57d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:04.719402 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-01948da2-01a3-498a-859c-6b75e328b57d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:04.730109 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a61f9-33f5-4f79-980e-bca859c3580f None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:04.802189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01948da2-01a3-498a-859c-6b75e328b57d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:04.803767 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01948da2-01a3-498a-859c-6b75e328b57d service swift] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:04.815709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 707/1399] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 19 10:29:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 10:29:04.845848 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:04.848197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101da76-3755-491c-a798-187b0ce10da7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:04.848810 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101da76-3755-491c-a798-187b0ce10da7 service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:04.849116 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101da76-3755-491c-a798-187b0ce10da7 service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:04.849727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101da76-3755-491c-a798-187b0ce10da7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:04.851419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-6101da76-3755-491c-a798-187b0ce10da7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:04.941308 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101da76-3755-491c-a798-187b0ce10da7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:04.943387 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101da76-3755-491c-a798-187b0ce10da7 service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:04.961387 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 693/1400] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:29:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:29:07.783037 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00e20a-6e94-47f4-80d3-39211ccafdb6 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:07.783638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00e20a-6e94-47f4-80d3-39211ccafdb6 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:07.783638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00e20a-6e94-47f4-80d3-39211ccafdb6 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:07.785136 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 708/1401] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:07.790886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937a88d-79bb-4738-afae-40a7649a446b None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:07.791341 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937a88d-79bb-4738-afae-40a7649a446b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:07.791441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937a88d-79bb-4738-afae-40a7649a446b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:07.873570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-8937a88d-79bb-4738-afae-40a7649a446b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:07.874914 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-8937a88d-79bb-4738-afae-40a7649a446b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:07.948399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:07.989749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 694/1402] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:08.277485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00e20a-6e94-47f4-80d3-39211ccafdb6 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:08.279031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6f00e20a-6e94-47f4-80d3-39211ccafdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:08.350592 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:08.448891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00e20a-6e94-47f4-80d3-39211ccafdb6 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:08.579013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee5bae-0ebc-454c-8e71-7bc3164be170 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:08.579629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee5bae-0ebc-454c-8e71-7bc3164be170 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:08.579920 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee5bae-0ebc-454c-8e71-7bc3164be170 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:08.580216 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee5bae-0ebc-454c-8e71-7bc3164be170 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:08.581915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f3ee5bae-0ebc-454c-8e71-7bc3164be170 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:08.602593 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:08.678160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee5bae-0ebc-454c-8e71-7bc3164be170 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:08.679760 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee5bae-0ebc-454c-8e71-7bc3164be170 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:08.700816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 709/1403] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:29:10.995242 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b72f46-ad60-4010-a85c-2b37a7b3acc9 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:10.995765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b72f46-ad60-4010-a85c-2b37a7b3acc9 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:10.995765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b72f46-ad60-4010-a85c-2b37a7b3acc9 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:10.997282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 695/1404] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:11.002417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f7b98-3e44-4312-963c-0d85d29e948b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:11.002848 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f7b98-3e44-4312-963c-0d85d29e948b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:11.002952 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f7b98-3e44-4312-963c-0d85d29e948b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:11.081846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-422f7b98-3e44-4312-963c-0d85d29e948b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:11.086956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-422f7b98-3e44-4312-963c-0d85d29e948b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:11.186114 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:11.196969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 710/1405] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:11.487153 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b72f46-ad60-4010-a85c-2b37a7b3acc9 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:11.489218 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-a9b72f46-ad60-4010-a85c-2b37a7b3acc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:11.654253 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b72f46-ad60-4010-a85c-2b37a7b3acc9 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:11.719806 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:11.781765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e274f-0060-46be-b97f-95001ae6b2e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:11.782352 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e274f-0060-46be-b97f-95001ae6b2e5 service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:11.782750 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e274f-0060-46be-b97f-95001ae6b2e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:11.783041 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e274f-0060-46be-b97f-95001ae6b2e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:11.784658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d38e274f-0060-46be-b97f-95001ae6b2e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:11.899600 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e274f-0060-46be-b97f-95001ae6b2e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:11.901292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e274f-0060-46be-b97f-95001ae6b2e5 service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:11.923886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 696/1406] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:29:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:29:14.253327 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a3ee1-8f68-4aae-b6d7-54ddef6a0f99 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:14.253942 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a3ee1-8f68-4aae-b6d7-54ddef6a0f99 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:14.253942 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a3ee1-8f68-4aae-b6d7-54ddef6a0f99 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:14.255407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 711/1407] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:14.260687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25417cf5-7e21-4c1a-ae21-ecc500a624fa None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:14.261185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25417cf5-7e21-4c1a-ae21-ecc500a624fa None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:14.262575 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25417cf5-7e21-4c1a-ae21-ecc500a624fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:14.336602 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-25417cf5-7e21-4c1a-ae21-ecc500a624fa None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:14.337909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-25417cf5-7e21-4c1a-ae21-ecc500a624fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:14.353979 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:14.449490 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 697/1408] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:29:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:14.738415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a3ee1-8f68-4aae-b6d7-54ddef6a0f99 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:14.739938 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f39a3ee1-8f68-4aae-b6d7-54ddef6a0f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:14.898775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a3ee1-8f68-4aae-b6d7-54ddef6a0f99 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:14.972074 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:15.024724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13598b-3f22-44b1-ae28-1b078c410f3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:15.025658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13598b-3f22-44b1-ae28-1b078c410f3a service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:15.026119 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13598b-3f22-44b1-ae28-1b078c410f3a service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:15.026399 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13598b-3f22-44b1-ae28-1b078c410f3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:15.028254 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bf13598b-3f22-44b1-ae28-1b078c410f3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:15.130348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13598b-3f22-44b1-ae28-1b078c410f3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:15.131966 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13598b-3f22-44b1-ae28-1b078c410f3a service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:15.153472 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 712/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:29:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:29:24.445440 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=38 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:25.150580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=64 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:29.596296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42931560-b7ff-4ff7-a713-2aff8c73b71c None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:29.596838 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42931560-b7ff-4ff7-a713-2aff8c73b71c None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:29.596838 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42931560-b7ff-4ff7-a713-2aff8c73b71c None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:29.598657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 698/1410] 10.222.0.42 () {54 vars in 918 bytes} [Mon Sep 19 10:29:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:29.603646 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f7e47-b8c0-4cf8-979a-ee730d0db6e3 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:29.604091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f7e47-b8c0-4cf8-979a-ee730d0db6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:29.604195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f7e47-b8c0-4cf8-979a-ee730d0db6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:29.682101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-2e8f7e47-b8c0-4cf8-979a-ee730d0db6e3 None None] MFA Rules not processed for user `ce0ef47a60f54ca1853fd267e308c58b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:29.683291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2e8f7e47-b8c0-4cf8-979a-ee730d0db6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:29.784160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 713/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Sep 19 10:29:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:29.798830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e710588-ce67-41f7-8a1f-670cab247bdc None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:29.799130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e710588-ce67-41f7-8a1f-670cab247bdc None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:29.799406 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e710588-ce67-41f7-8a1f-670cab247bdc None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:29.801379 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 699/1412] 10.222.0.42 () {54 vars in 918 bytes} [Mon Sep 19 10:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:29:29.805956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7623b7b-8652-4b46-8063-bbe6d812a4be None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:29.806264 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7623b7b-8652-4b46-8063-bbe6d812a4be None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:29.806536 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7623b7b-8652-4b46-8063-bbe6d812a4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:29.880408 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:29.882884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-d7623b7b-8652-4b46-8063-bbe6d812a4be None None] MFA Rules not processed for user `b4007794a30247c4a318af62d05e3b92`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:29.884081 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d7623b7b-8652-4b46-8063-bbe6d812a4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:29.997088 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 714/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Sep 19 10:29:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:32.591940 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfb493-09bb-4f74-a3ed-14c508894c22 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:32.592514 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfb493-09bb-4f74-a3ed-14c508894c22 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:32.592514 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfb493-09bb-4f74-a3ed-14c508894c22 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:32.593852 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 10:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:32.599597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44190ffb-d463-469e-9d48-759c70c96e04 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:32.600034 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44190ffb-d463-469e-9d48-759c70c96e04 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:32.600146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44190ffb-d463-469e-9d48-759c70c96e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:32.665890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-44190ffb-d463-469e-9d48-759c70c96e04 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:32.667117 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-44190ffb-d463-469e-9d48-759c70c96e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:32.719974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:32.762292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 10:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:33.050017 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfb493-09bb-4f74-a3ed-14c508894c22 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:33.051541 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-20bfb493-09bb-4f74-a3ed-14c508894c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:33.194997 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfb493-09bb-4f74-a3ed-14c508894c22 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:33.265353 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:33.306868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bfde3-469b-412b-a165-59673bfcd376 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:33.307402 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bfde3-469b-412b-a165-59673bfcd376 service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:33.307682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bfde3-469b-412b-a165-59673bfcd376 service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:33.307959 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bfde3-469b-412b-a165-59673bfcd376 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:33.309625 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4f1bfde3-469b-412b-a165-59673bfcd376 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:33.398483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1bfde3-469b-412b-a165-59673bfcd376 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:33.400078 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1bfde3-469b-412b-a165-59673bfcd376 service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:33.417624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 701/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:29:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:29:34.557522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9803b0c-e6eb-4627-bfc7-196b84220956 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:34.558129 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9803b0c-e6eb-4627-bfc7-196b84220956 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:34.558129 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9803b0c-e6eb-4627-bfc7-196b84220956 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:34.559454 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 10:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:34.566226 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b1edf-27cd-4369-976a-8c1a455b90b5 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:34.566645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b1edf-27cd-4369-976a-8c1a455b90b5 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:34.566776 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b1edf-27cd-4369-976a-8c1a455b90b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:34.640772 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-0e6b1edf-27cd-4369-976a-8c1a455b90b5 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:34.641990 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-0e6b1edf-27cd-4369-976a-8c1a455b90b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:34.667998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:34.737902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 10:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:34.887264 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9803b0c-e6eb-4627-bfc7-196b84220956 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:34.888767 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c9803b0c-e6eb-4627-bfc7-196b84220956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:35.031871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9803b0c-e6eb-4627-bfc7-196b84220956 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:35.072558 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:35.143625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709429b8-883e-4b09-89bd-75032cad2022 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:35.144144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709429b8-883e-4b09-89bd-75032cad2022 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:35.144416 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709429b8-883e-4b09-89bd-75032cad2022 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:35.144680 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709429b8-883e-4b09-89bd-75032cad2022 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:35.146279 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-709429b8-883e-4b09-89bd-75032cad2022 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:35.237014 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709429b8-883e-4b09-89bd-75032cad2022 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:35.238694 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709429b8-883e-4b09-89bd-75032cad2022 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:35.256632 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 717/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:29:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:29:36.388749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5349e-652c-4383-ba97-837f4d42b8d2 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:36.389351 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5349e-652c-4383-ba97-837f4d42b8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:36.389351 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5349e-652c-4383-ba97-837f4d42b8d2 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:36.390673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 10:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:36.397053 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f3d2c-3a3c-4c29-89ed-c7ea19df2539 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:36.397524 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f3d2c-3a3c-4c29-89ed-c7ea19df2539 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:36.397632 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f3d2c-3a3c-4c29-89ed-c7ea19df2539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:36.452552 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:36.474501 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6f2f3d2c-3a3c-4c29-89ed-c7ea19df2539 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:36.475731 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6f2f3d2c-3a3c-4c29-89ed-c7ea19df2539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:36.570336 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 10:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:36.720798 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5349e-652c-4383-ba97-837f4d42b8d2 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:36.723028 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-6dc5349e-652c-4383-ba97-837f4d42b8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:36.866294 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5349e-652c-4383-ba97-837f4d42b8d2 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:36.878236 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:36.979480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fefff-6452-4c3c-a297-a019000ce406 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:36.980014 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fefff-6452-4c3c-a297-a019000ce406 service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:36.980290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fefff-6452-4c3c-a297-a019000ce406 service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:36.980589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fefff-6452-4c3c-a297-a019000ce406 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:36.982206 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-af8fefff-6452-4c3c-a297-a019000ce406 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:37.069703 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fefff-6452-4c3c-a297-a019000ce406 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:37.071297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fefff-6452-4c3c-a297-a019000ce406 service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:37.089484 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 704/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:29:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:29:38.215958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaca10-f5b0-4b79-bf81-4a13af6e04eb None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:38.216456 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaca10-f5b0-4b79-bf81-4a13af6e04eb None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:38.216590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaca10-f5b0-4b79-bf81-4a13af6e04eb None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:38.217890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 10:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:38.225160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228951c-4034-4b0d-a892-5366e56638e7 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:38.225626 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228951c-4034-4b0d-a892-5366e56638e7 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:38.225745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228951c-4034-4b0d-a892-5366e56638e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:38.246122 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:38.303347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-2228951c-4034-4b0d-a892-5366e56638e7 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:38.304569 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-2228951c-4034-4b0d-a892-5366e56638e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:38.399252 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 10:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:38.549361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaca10-f5b0-4b79-bf81-4a13af6e04eb None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:38.550848 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cecaca10-f5b0-4b79-bf81-4a13af6e04eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:38.659400 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:38.694852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaca10-f5b0-4b79-bf81-4a13af6e04eb None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:38.806067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46301c36-4a04-4cb8-a23c-859a3054d582 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:38.806598 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46301c36-4a04-4cb8-a23c-859a3054d582 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:38.806878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46301c36-4a04-4cb8-a23c-859a3054d582 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:38.807151 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46301c36-4a04-4cb8-a23c-859a3054d582 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:38.808730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-46301c36-4a04-4cb8-a23c-859a3054d582 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:38.885020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:38.898724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46301c36-4a04-4cb8-a23c-859a3054d582 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:38.900311 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46301c36-4a04-4cb8-a23c-859a3054d582 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:38.918791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 720/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:29:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:29:40.029044 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4eb54-e841-48c7-9e26-88e76ce2d6a6 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:40.029604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4eb54-e841-48c7-9e26-88e76ce2d6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:40.029604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4eb54-e841-48c7-9e26-88e76ce2d6a6 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:40.030962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 10:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:40.037580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401ded9-3f94-454b-817a-102c4f3e1f2a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:40.038013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401ded9-3f94-454b-817a-102c4f3e1f2a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:40.038175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401ded9-3f94-454b-817a-102c4f3e1f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:40.114898 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5401ded9-3f94-454b-817a-102c4f3e1f2a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:40.116025 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5401ded9-3f94-454b-817a-102c4f3e1f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:40.215686 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 10:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:40.367264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4eb54-e841-48c7-9e26-88e76ce2d6a6 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:40.368708 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e8f4eb54-e841-48c7-9e26-88e76ce2d6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:40.455719 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:40.519768 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4eb54-e841-48c7-9e26-88e76ce2d6a6 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:40.631025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e3e1c-67d2-420d-a53a-3eb7ddc2792b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:40.631552 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e3e1c-67d2-420d-a53a-3eb7ddc2792b service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:40.631827 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e3e1c-67d2-420d-a53a-3eb7ddc2792b service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:40.632099 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e3e1c-67d2-420d-a53a-3eb7ddc2792b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:40.633709 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-930e3e1c-67d2-420d-a53a-3eb7ddc2792b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:40.681695 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:40.721391 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e3e1c-67d2-420d-a53a-3eb7ddc2792b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:40.722976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e3e1c-67d2-420d-a53a-3eb7ddc2792b service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:40.740743 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 707/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:29:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:29:41.839636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c420d1e-1df5-4c18-8da1-d80f9cf764dd None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:41.840065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c420d1e-1df5-4c18-8da1-d80f9cf764dd None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:41.840187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c420d1e-1df5-4c18-8da1-d80f9cf764dd None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:41.841563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 10:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:41.847780 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15111a-e506-4e83-8b61-5940097397a0 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:41.848196 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15111a-e506-4e83-8b61-5940097397a0 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:41.848311 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15111a-e506-4e83-8b61-5940097397a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:41.921926 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-0f15111a-e506-4e83-8b61-5940097397a0 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:41.923121 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-0f15111a-e506-4e83-8b61-5940097397a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:42.017715 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 10:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:42.166899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c420d1e-1df5-4c18-8da1-d80f9cf764dd None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:42.168400 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3c420d1e-1df5-4c18-8da1-d80f9cf764dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:42.217226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:42.312939 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c420d1e-1df5-4c18-8da1-d80f9cf764dd None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:42.422828 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a9a6f-2b6a-41f0-9d96-32fdefac31e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:42.423350 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a9a6f-2b6a-41f0-9d96-32fdefac31e7 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:42.423625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a9a6f-2b6a-41f0-9d96-32fdefac31e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:42.423898 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a9a6f-2b6a-41f0-9d96-32fdefac31e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:42.426168 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3c6a9a6f-2b6a-41f0-9d96-32fdefac31e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:42.432899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:42.514095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a9a6f-2b6a-41f0-9d96-32fdefac31e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:42.515689 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a9a6f-2b6a-41f0-9d96-32fdefac31e7 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:42.533653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 723/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:29:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:29:43.652830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99dfbd-43a9-43b2-a7c8-61741dcd2cdb None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:43.653430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99dfbd-43a9-43b2-a7c8-61741dcd2cdb None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:43.653430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99dfbd-43a9-43b2-a7c8-61741dcd2cdb None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:43.654784 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 10:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:43.661021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790d979-5569-44c4-b9ff-1dd36cd3d315 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:43.661509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790d979-5569-44c4-b9ff-1dd36cd3d315 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:43.661748 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790d979-5569-44c4-b9ff-1dd36cd3d315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:43.739801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e790d979-5569-44c4-b9ff-1dd36cd3d315 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:43.741011 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e790d979-5569-44c4-b9ff-1dd36cd3d315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:43.808972 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:43.838274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 10:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:43.989087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99dfbd-43a9-43b2-a7c8-61741dcd2cdb None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:43.990592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-aa99dfbd-43a9-43b2-a7c8-61741dcd2cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:44.008046 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:44.135420 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99dfbd-43a9-43b2-a7c8-61741dcd2cdb None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:44.224951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:44.250604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5f41f-8610-4829-8696-1cb6b55582b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:44.251135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5f41f-8610-4829-8696-1cb6b55582b4 service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:44.251412 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5f41f-8610-4829-8696-1cb6b55582b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:44.251734 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5f41f-8610-4829-8696-1cb6b55582b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:44.253975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-50f5f41f-8610-4829-8696-1cb6b55582b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:44.341820 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5f41f-8610-4829-8696-1cb6b55582b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:44.343420 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5f41f-8610-4829-8696-1cb6b55582b4 service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:44.361417 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 710/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:29:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:29:45.513147 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4103ce-ff47-437a-b3b9-428b775b27e6 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:45.513758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4103ce-ff47-437a-b3b9-428b775b27e6 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:45.513758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4103ce-ff47-437a-b3b9-428b775b27e6 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:45.515017 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 10:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:45.523130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d1978-251d-42f4-af97-1a07ff9722c2 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:45.523638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d1978-251d-42f4-af97-1a07ff9722c2 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:45.523749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d1978-251d-42f4-af97-1a07ff9722c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:45.598902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-3a9d1978-251d-42f4-af97-1a07ff9722c2 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:45.600115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-3a9d1978-251d-42f4-af97-1a07ff9722c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:45.640357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:45.694180 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 10:29:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:45.844303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4103ce-ff47-437a-b3b9-428b775b27e6 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:45.845856 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1c4103ce-ff47-437a-b3b9-428b775b27e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:45.988954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4103ce-ff47-437a-b3b9-428b775b27e6 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:46.041425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:46.098547 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530727e9-4ab1-46ad-ba96-0fd9cc1d7753 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:46.099072 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530727e9-4ab1-46ad-ba96-0fd9cc1d7753 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:46.099347 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530727e9-4ab1-46ad-ba96-0fd9cc1d7753 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:46.099643 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530727e9-4ab1-46ad-ba96-0fd9cc1d7753 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:46.101336 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-530727e9-4ab1-46ad-ba96-0fd9cc1d7753 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:46.188600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530727e9-4ab1-46ad-ba96-0fd9cc1d7753 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:46.190701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530727e9-4ab1-46ad-ba96-0fd9cc1d7753 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:46.208948 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 726/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:29:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:29:47.369406 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711f090-968d-40de-87d9-81238d15a893 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:47.370104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711f090-968d-40de-87d9-81238d15a893 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:47.370104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711f090-968d-40de-87d9-81238d15a893 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:47.371303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 10:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:47.377867 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073c603-9d08-4152-9cd3-e967df5bb81b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:47.378358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073c603-9d08-4152-9cd3-e967df5bb81b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:47.378461 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073c603-9d08-4152-9cd3-e967df5bb81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:47.454249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-3073c603-9d08-4152-9cd3-e967df5bb81b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:47.455951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3073c603-9d08-4152-9cd3-e967df5bb81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:47.458010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:47.556699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 10:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:47.706899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711f090-968d-40de-87d9-81238d15a893 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:47.708449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-3711f090-968d-40de-87d9-81238d15a893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:47.851756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711f090-968d-40de-87d9-81238d15a893 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:47.880409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:47.966520 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3772b7b-3724-4f72-be97-4705b638b1a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:47.967044 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3772b7b-3724-4f72-be97-4705b638b1a4 service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:47.967322 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3772b7b-3724-4f72-be97-4705b638b1a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:47.967593 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3772b7b-3724-4f72-be97-4705b638b1a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:47.969213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c3772b7b-3724-4f72-be97-4705b638b1a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:48.058918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3772b7b-3724-4f72-be97-4705b638b1a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:48.060529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3772b7b-3724-4f72-be97-4705b638b1a4 service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:48.078860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 713/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:29:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:29:50.138191 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcd691-d90f-4d81-8bbd-780650ed794e None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:50.138811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcd691-d90f-4d81-8bbd-780650ed794e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:50.138811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcd691-d90f-4d81-8bbd-780650ed794e None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:50.140011 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 10:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:50.145290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f808d7-fb4c-4af3-80b0-03a567d9434c None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:50.145721 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f808d7-fb4c-4af3-80b0-03a567d9434c None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:50.145828 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f808d7-fb4c-4af3-80b0-03a567d9434c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:50.187011 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:50.212420 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-97f808d7-fb4c-4af3-80b0-03a567d9434c None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:50.213672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-97f808d7-fb4c-4af3-80b0-03a567d9434c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:50.310625 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 10:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:50.595403 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcd691-d90f-4d81-8bbd-780650ed794e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:50.598808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-40bcd691-d90f-4d81-8bbd-780650ed794e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:50.699251 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcd691-d90f-4d81-8bbd-780650ed794e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:50.731186 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:50.772349 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d94580-bd31-4efe-9e0f-2e6010835584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:50.772942 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d94580-bd31-4efe-9e0f-2e6010835584 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:50.773250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d94580-bd31-4efe-9e0f-2e6010835584 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:50.773518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d94580-bd31-4efe-9e0f-2e6010835584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:50.775076 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-75d94580-bd31-4efe-9e0f-2e6010835584 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:50.881573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d94580-bd31-4efe-9e0f-2e6010835584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:50.882910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d94580-bd31-4efe-9e0f-2e6010835584 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:50.902470 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 729/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:29:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:29:53.193804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ff213-483b-4de3-b5cb-607033799b01 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:53.194353 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ff213-483b-4de3-b5cb-607033799b01 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:53.194353 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ff213-483b-4de3-b5cb-607033799b01 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:53.195698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 10:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:53.201014 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f59b0-43b3-4593-bdec-fd07eca47a2e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:53.201444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f59b0-43b3-4593-bdec-fd07eca47a2e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:53.201595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f59b0-43b3-4593-bdec-fd07eca47a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:53.266407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-bf0f59b0-43b3-4593-bdec-fd07eca47a2e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:53.267614 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bf0f59b0-43b3-4593-bdec-fd07eca47a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:53.280894 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:53.364250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 10:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:53.645994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ff213-483b-4de3-b5cb-607033799b01 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:53.647628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-513ff213-483b-4de3-b5cb-607033799b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:53.750231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ff213-483b-4de3-b5cb-607033799b01 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:53.763524 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:53.825880 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c280bb-fb88-44d4-bfd5-84e3c93f7a5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:53.826494 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c280bb-fb88-44d4-bfd5-84e3c93f7a5b service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:53.826795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c280bb-fb88-44d4-bfd5-84e3c93f7a5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:53.827068 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c280bb-fb88-44d4-bfd5-84e3c93f7a5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:53.828679 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-92c280bb-fb88-44d4-bfd5-84e3c93f7a5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:53.936310 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c280bb-fb88-44d4-bfd5-84e3c93f7a5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:53.937796 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c280bb-fb88-44d4-bfd5-84e3c93f7a5b service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:53.955672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 716/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:29:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:29:56.558967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f0eda-ebed-4528-828d-9fd01cca94fa None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:56.559540 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f0eda-ebed-4528-828d-9fd01cca94fa None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:56.559540 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f0eda-ebed-4528-828d-9fd01cca94fa None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:56.560886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 10:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:56.566788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b319b-2da3-41a9-9c1f-4a7b6c4f1093 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:56.567229 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b319b-2da3-41a9-9c1f-4a7b6c4f1093 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:56.567336 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b319b-2da3-41a9-9c1f-4a7b6c4f1093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:56.614281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:56.632041 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-996b319b-2da3-41a9-9c1f-4a7b6c4f1093 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:56.633294 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-996b319b-2da3-41a9-9c1f-4a7b6c4f1093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:56.728051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 10:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:29:57.018170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f0eda-ebed-4528-828d-9fd01cca94fa None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:29:57.019722 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d34f0eda-ebed-4528-828d-9fd01cca94fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:57.121278 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f0eda-ebed-4528-828d-9fd01cca94fa None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:29:57.164726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:57.198792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32883aa-363c-4674-a8c9-d39e69f372ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:29:57.199402 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32883aa-363c-4674-a8c9-d39e69f372ab service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:57.199681 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32883aa-363c-4674-a8c9-d39e69f372ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:57.199956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32883aa-363c-4674-a8c9-d39e69f372ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:57.201584 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f32883aa-363c-4674-a8c9-d39e69f372ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:57.309716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32883aa-363c-4674-a8c9-d39e69f372ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:29:57.311143 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32883aa-363c-4674-a8c9-d39e69f372ab service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:29:57.330788 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 732/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:29:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:29:59.587303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d34540-2732-46fa-ba12-31de615c31d1 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:59.587871 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d34540-2732-46fa-ba12-31de615c31d1 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:59.587871 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d34540-2732-46fa-ba12-31de615c31d1 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:59.589197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 10:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:29:59.594525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ba67d-7973-465c-b35d-dca5d3ea2dc7 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:29:59.594929 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ba67d-7973-465c-b35d-dca5d3ea2dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:29:59.595080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ba67d-7973-465c-b35d-dca5d3ea2dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:29:59.659676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-9f0ba67d-7973-465c-b35d-dca5d3ea2dc7 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:29:59.660895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9f0ba67d-7973-465c-b35d-dca5d3ea2dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:29:59.681161 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:29:59.756567 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 10:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:30:00.045986 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d34540-2732-46fa-ba12-31de615c31d1 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:30:00.047522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-88d34540-2732-46fa-ba12-31de615c31d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:00.150938 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d34540-2732-46fa-ba12-31de615c31d1 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:30:00.170860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:00.223009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3b91a-43bb-4843-b94b-129255e50c49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:30:00.223637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3b91a-43bb-4843-b94b-129255e50c49 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:00.223915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3b91a-43bb-4843-b94b-129255e50c49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:00.224185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3b91a-43bb-4843-b94b-129255e50c49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:00.225809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-17d3b91a-43bb-4843-b94b-129255e50c49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:00.340144 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3b91a-43bb-4843-b94b-129255e50c49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:30:00.341635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3b91a-43bb-4843-b94b-129255e50c49 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:30:00.359608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 719/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:30:02.633755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddde30-0ba1-4c58-ab1b-3c59f7a66899 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:02.634322 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddde30-0ba1-4c58-ab1b-3c59f7a66899 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:02.634322 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddde30-0ba1-4c58-ab1b-3c59f7a66899 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:02.635658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 10:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:30:02.640922 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5767e4f-56ca-4f6a-94bb-1dea62655dc6 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:02.641385 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5767e4f-56ca-4f6a-94bb-1dea62655dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:02.641503 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5767e4f-56ca-4f6a-94bb-1dea62655dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:02.704455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:02.707724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-b5767e4f-56ca-4f6a-94bb-1dea62655dc6 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:30:02.708921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b5767e4f-56ca-4f6a-94bb-1dea62655dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:02.804068 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 10:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:30:03.088532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddde30-0ba1-4c58-ab1b-3c59f7a66899 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:30:03.090111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9fddde30-0ba1-4c58-ab1b-3c59f7a66899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:03.192564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddde30-0ba1-4c58-ab1b-3c59f7a66899 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:30:03.241200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:03.265091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039c6cf-96a0-4349-ad78-d2a5c0bfad67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:30:03.265713 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039c6cf-96a0-4349-ad78-d2a5c0bfad67 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:03.265988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039c6cf-96a0-4349-ad78-d2a5c0bfad67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:03.266255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039c6cf-96a0-4349-ad78-d2a5c0bfad67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:03.267885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1039c6cf-96a0-4349-ad78-d2a5c0bfad67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:03.379663 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039c6cf-96a0-4349-ad78-d2a5c0bfad67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:30:03.381012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039c6cf-96a0-4349-ad78-d2a5c0bfad67 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:30:03.398480 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 735/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:30:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:30:05.604731 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669de8c-bb58-4bd6-930c-01b722b4f2bc None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:05.605296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669de8c-bb58-4bd6-930c-01b722b4f2bc None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:05.605296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669de8c-bb58-4bd6-930c-01b722b4f2bc None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:05.606553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 10:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:30:05.611880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6b302-a485-48ee-b68f-94efeb14f4d3 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:05.612281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6b302-a485-48ee-b68f-94efeb14f4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:05.612440 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6b302-a485-48ee-b68f-94efeb14f4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:05.677584 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-afe6b302-a485-48ee-b68f-94efeb14f4d3 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:30:05.678785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-afe6b302-a485-48ee-b68f-94efeb14f4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:05.707864 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:05.774075 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 10:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:30:06.061176 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669de8c-bb58-4bd6-930c-01b722b4f2bc None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:30:06.062742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c669de8c-bb58-4bd6-930c-01b722b4f2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:06.164219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669de8c-bb58-4bd6-930c-01b722b4f2bc None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:30:06.193310 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:06.238611 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bd36f-bf72-4dbd-8d1c-602a25a0b462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:30:06.239215 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bd36f-bf72-4dbd-8d1c-602a25a0b462 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:06.239495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bd36f-bf72-4dbd-8d1c-602a25a0b462 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:06.239766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bd36f-bf72-4dbd-8d1c-602a25a0b462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:06.241404 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-a88bd36f-bf72-4dbd-8d1c-602a25a0b462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:06.352319 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88bd36f-bf72-4dbd-8d1c-602a25a0b462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:30:06.353723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88bd36f-bf72-4dbd-8d1c-602a25a0b462 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:30:06.372228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 722/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:30:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:30:08.654644 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d61e6f-ce0e-469e-ad80-17a7e3645ca0 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:08.655157 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d61e6f-ce0e-469e-ad80-17a7e3645ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:08.655157 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d61e6f-ce0e-469e-ad80-17a7e3645ca0 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:08.656450 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 10:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:30:08.662376 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c1462-c334-452d-8615-8c5fe9f16908 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:08.662829 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c1462-c334-452d-8615-8c5fe9f16908 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:08.662942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c1462-c334-452d-8615-8c5fe9f16908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:08.730262 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-ea3c1462-c334-452d-8615-8c5fe9f16908 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:30:08.731475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-ea3c1462-c334-452d-8615-8c5fe9f16908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:08.740295 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:08.827090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 10:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:30:09.114146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d61e6f-ce0e-469e-ad80-17a7e3645ca0 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:30:09.115693 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-93d61e6f-ce0e-469e-ad80-17a7e3645ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:09.219202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d61e6f-ce0e-469e-ad80-17a7e3645ca0 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:30:09.273754 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:09.292253 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436d605-f881-4544-a5a2-79b16cf77115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:30:09.292845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436d605-f881-4544-a5a2-79b16cf77115 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:09.293148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436d605-f881-4544-a5a2-79b16cf77115 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:09.293421 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436d605-f881-4544-a5a2-79b16cf77115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:09.295088 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2436d605-f881-4544-a5a2-79b16cf77115 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:09.405749 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436d605-f881-4544-a5a2-79b16cf77115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:30:09.407081 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436d605-f881-4544-a5a2-79b16cf77115 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:30:09.426146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 738/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:30:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:30:11.618101 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a70bf9-e851-48ae-9466-0f6d687624d4 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:11.618642 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a70bf9-e851-48ae-9466-0f6d687624d4 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:11.618764 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a70bf9-e851-48ae-9466-0f6d687624d4 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:11.620031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 10:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:30:11.625207 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83898a8-4b70-4f80-8117-27d33695ce3c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:11.625617 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83898a8-4b70-4f80-8117-27d33695ce3c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:11.625750 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83898a8-4b70-4f80-8117-27d33695ce3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:11.689672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e83898a8-4b70-4f80-8117-27d33695ce3c None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:30:11.690864 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e83898a8-4b70-4f80-8117-27d33695ce3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:11.727503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:11.785455 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 10:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:30:12.066972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a70bf9-e851-48ae-9466-0f6d687624d4 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:30:12.068529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-46a70bf9-e851-48ae-9466-0f6d687624d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:12.171118 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a70bf9-e851-48ae-9466-0f6d687624d4 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:30:12.208208 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:12.242963 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46bf9e-3fbc-4b33-b346-6e9c6062895c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:30:12.243565 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46bf9e-3fbc-4b33-b346-6e9c6062895c service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:12.243845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46bf9e-3fbc-4b33-b346-6e9c6062895c service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:12.244119 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46bf9e-3fbc-4b33-b346-6e9c6062895c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:12.246651 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-dc46bf9e-3fbc-4b33-b346-6e9c6062895c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:12.364182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc46bf9e-3fbc-4b33-b346-6e9c6062895c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:30:12.365638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc46bf9e-3fbc-4b33-b346-6e9c6062895c service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:30:12.384221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 725/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:30:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:30:14.543746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffc13b-4f8d-46a5-b532-22b47e8a34d2 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:14.544273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffc13b-4f8d-46a5-b532-22b47e8a34d2 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:14.544273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffc13b-4f8d-46a5-b532-22b47e8a34d2 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:14.545640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 10:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:30:14.551669 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1e6f7-244c-4cf9-9401-0d8d0963b163 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:14.552082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1e6f7-244c-4cf9-9401-0d8d0963b163 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:14.552221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1e6f7-244c-4cf9-9401-0d8d0963b163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:14.619302 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-26d1e6f7-244c-4cf9-9401-0d8d0963b163 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:30:14.620508 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-26d1e6f7-244c-4cf9-9401-0d8d0963b163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:14.638051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:14.717015 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 10:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:30:14.999280 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffc13b-4f8d-46a5-b532-22b47e8a34d2 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:30:15.000840 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-54ffc13b-4f8d-46a5-b532-22b47e8a34d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:15.104541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffc13b-4f8d-46a5-b532-22b47e8a34d2 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:30:15.170094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:15.176530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b5c9b-e6fb-4e39-adb1-fd23147685e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:30:15.177013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b5c9b-e6fb-4e39-adb1-fd23147685e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:15.177316 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b5c9b-e6fb-4e39-adb1-fd23147685e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:15.177588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b5c9b-e6fb-4e39-adb1-fd23147685e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:15.179164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5f1b5c9b-e6fb-4e39-adb1-fd23147685e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:15.287677 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b5c9b-e6fb-4e39-adb1-fd23147685e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:30:15.289130 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b5c9b-e6fb-4e39-adb1-fd23147685e9 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:30:15.307030 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 741/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:30:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:30:17.475766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23113e6a-7374-4648-b121-cf34b2d5c330 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:17.476311 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23113e6a-7374-4648-b121-cf34b2d5c330 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:17.476311 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23113e6a-7374-4648-b121-cf34b2d5c330 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:17.477670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 10:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:30:17.482980 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577eba9c-9e25-4dcc-aca3-72d970bad75e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:17.483408 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577eba9c-9e25-4dcc-aca3-72d970bad75e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:17.483530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577eba9c-9e25-4dcc-aca3-72d970bad75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:17.551951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-577eba9c-9e25-4dcc-aca3-72d970bad75e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:30:17.553189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-577eba9c-9e25-4dcc-aca3-72d970bad75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:17.601095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:17.648681 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 10:30:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:30:17.936478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23113e6a-7374-4648-b121-cf34b2d5c330 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:30:17.938076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-23113e6a-7374-4648-b121-cf34b2d5c330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:18.044123 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23113e6a-7374-4648-b121-cf34b2d5c330 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:30:18.093760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:18.118810 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be483946-1bd0-4550-add4-cc8f65292ae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:30:18.119448 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be483946-1bd0-4550-add4-cc8f65292ae8 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:18.119724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be483946-1bd0-4550-add4-cc8f65292ae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:18.119991 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be483946-1bd0-4550-add4-cc8f65292ae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:18.121592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-be483946-1bd0-4550-add4-cc8f65292ae8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:18.229520 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be483946-1bd0-4550-add4-cc8f65292ae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:30:18.231003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be483946-1bd0-4550-add4-cc8f65292ae8 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:30:18.249032 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 728/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:30:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:30:27.647164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=22 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:28.246848 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=66 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:38.214867 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15da31-b7f8-4101-b75b-ef7b196e9d45 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:38.215409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15da31-b7f8-4101-b75b-ef7b196e9d45 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:38.215409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15da31-b7f8-4101-b75b-ef7b196e9d45 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:38.216873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 743/1471] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:30:38.222452 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3a2d1-b63c-4c6c-b622-6388381a03a4 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:38.222878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3a2d1-b63c-4c6c-b622-6388381a03a4 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:38.222996 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3a2d1-b63c-4c6c-b622-6388381a03a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:38.299527 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-dcb3a2d1-b63c-4c6c-b622-6388381a03a4 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:30:38.300736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-dcb3a2d1-b63c-4c6c-b622-6388381a03a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:38.401280 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 729/1472] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:30:38.702496 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c19e34-a470-4ac6-9c61-e04a3ae92ac7 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:38.702805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c19e34-a470-4ac6-9c61-e04a3ae92ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:38.703081 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c19e34-a470-4ac6-9c61-e04a3ae92ac7 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:38.704433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 744/1473] 10.222.0.42 () {54 vars in 962 bytes} [Mon Sep 19 10:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:30:38.709721 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737458b8-26a8-45f4-8e8e-e29a697cb119 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:38.710007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737458b8-26a8-45f4-8e8e-e29a697cb119 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:38.710414 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737458b8-26a8-45f4-8e8e-e29a697cb119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:38.789537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-737458b8-26a8-45f4-8e8e-e29a697cb119 None None] MFA Rules not processed for user `a40b1ed93a6b4e5fb0e8f907627379a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:30:38.790752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-737458b8-26a8-45f4-8e8e-e29a697cb119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:38.813413 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:38.870844 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 730/1474] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Sep 19 10:30:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:30:38.879198 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c19e34-a470-4ac6-9c61-e04a3ae92ac7 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:30:38.880722 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-26c19e34-a470-4ac6-9c61-e04a3ae92ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:38.989354 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c19e34-a470-4ac6-9c61-e04a3ae92ac7 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:30:39.066016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc87cc2-a5c6-4ea8-a737-0fad975d733b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a40b1ed93a6b4e5fb0e8f907627379a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:30:39.066635 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc87cc2-a5c6-4ea8-a737-0fad975d733b service glance] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:39.066972 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc87cc2-a5c6-4ea8-a737-0fad975d733b service glance] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:39.067187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc87cc2-a5c6-4ea8-a737-0fad975d733b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:39.068793 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1fc87cc2-a5c6-4ea8-a737-0fad975d733b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:39.125648 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:39.182794 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc87cc2-a5c6-4ea8-a737-0fad975d733b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:30:39.184168 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc87cc2-a5c6-4ea8-a737-0fad975d733b service glance] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:30:39.207463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 745/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:30:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:30:41.463324 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea19867-a42f-4a60-862e-0692f23a6953 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:41.463896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea19867-a42f-4a60-862e-0692f23a6953 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:41.463896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea19867-a42f-4a60-862e-0692f23a6953 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:41.465251 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 731/1476] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:30:41.471085 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b809ff9-414a-4c33-b04e-48d7f4fc8e71 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:41.471521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b809ff9-414a-4c33-b04e-48d7f4fc8e71 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:41.471626 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b809ff9-414a-4c33-b04e-48d7f4fc8e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:41.541158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5b809ff9-414a-4c33-b04e-48d7f4fc8e71 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:30:41.542493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5b809ff9-414a-4c33-b04e-48d7f4fc8e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:41.640157 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 746/1477] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:30:41.917156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea19867-a42f-4a60-862e-0692f23a6953 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:30:41.918704 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-7ea19867-a42f-4a60-862e-0692f23a6953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:42.045994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea19867-a42f-4a60-862e-0692f23a6953 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:30:42.096510 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:42.141700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ad78f-d9b0-4655-8988-3dce70864ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:30:42.142359 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ad78f-d9b0-4655-8988-3dce70864ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:42.142638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ad78f-d9b0-4655-8988-3dce70864ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:42.142933 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ad78f-d9b0-4655-8988-3dce70864ab4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:42.143541 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ad78f-d9b0-4655-8988-3dce70864ab4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:30:42.145385 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: WARNING py.warnings [None req-8b9ad78f-d9b0-4655-8988-3dce70864ab4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 10:30:42.145385 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: warnings.warn(msg) Sep 19 10:30:42.145385 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: Sep 19 10:30:42.146530 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ad78f-d9b0-4655-8988-3dce70864ab4 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:30:42.167474 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 732/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Sep 19 10:30:41 2022] GET /identity/v3/projects => generated 2553 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:30:44.336198 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f617a-16f7-4010-b36f-b7a408b79677 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:44.336765 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f617a-16f7-4010-b36f-b7a408b79677 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:44.336765 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f617a-16f7-4010-b36f-b7a408b79677 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:44.338183 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 747/1479] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:30:44.344142 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b35366-00ec-4e2a-ba92-539f6ebc2c82 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:44.344568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b35366-00ec-4e2a-ba92-539f6ebc2c82 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:44.344666 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b35366-00ec-4e2a-ba92-539f6ebc2c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:44.410870 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-24b35366-00ec-4e2a-ba92-539f6ebc2c82 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:30:44.412095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-24b35366-00ec-4e2a-ba92-539f6ebc2c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:44.508630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 733/1480] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:30:44.803879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f617a-16f7-4010-b36f-b7a408b79677 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:30:44.805417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a22f617a-16f7-4010-b36f-b7a408b79677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:44.815966 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:44.951035 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f617a-16f7-4010-b36f-b7a408b79677 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:30:45.035126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:45.067662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c0f19-dffe-466c-9da5-94f71a967c47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:30:45.068188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c0f19-dffe-466c-9da5-94f71a967c47 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:45.068461 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c0f19-dffe-466c-9da5-94f71a967c47 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:45.068731 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c0f19-dffe-466c-9da5-94f71a967c47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:45.070339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ea9c0f19-dffe-466c-9da5-94f71a967c47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:45.158359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c0f19-dffe-466c-9da5-94f71a967c47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:30:45.159906 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c0f19-dffe-466c-9da5-94f71a967c47 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:30:45.177769 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 748/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:30:47.606641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ea48b-9280-4157-9a77-197ece162269 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:47.607232 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ea48b-9280-4157-9a77-197ece162269 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:47.607232 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ea48b-9280-4157-9a77-197ece162269 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:47.608552 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 734/1482] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:30:47.613956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788245a-f83a-4a9b-8f07-9de6ce34c1ee None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:47.614362 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788245a-f83a-4a9b-8f07-9de6ce34c1ee None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:47.614516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788245a-f83a-4a9b-8f07-9de6ce34c1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:47.681104 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4788245a-f83a-4a9b-8f07-9de6ce34c1ee None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:30:47.682318 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4788245a-f83a-4a9b-8f07-9de6ce34c1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:47.728677 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:47.778676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 749/1483] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:30:48.067483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ea48b-9280-4157-9a77-197ece162269 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:30:48.069074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-af6ea48b-9280-4157-9a77-197ece162269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:48.092439 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:48.172703 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ea48b-9280-4157-9a77-197ece162269 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:30:48.255058 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae97af-399c-4eb0-b3a2-91560174d915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:30:48.255654 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae97af-399c-4eb0-b3a2-91560174d915 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:48.255926 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae97af-399c-4eb0-b3a2-91560174d915 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:48.256193 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae97af-399c-4eb0-b3a2-91560174d915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:48.257889 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c1ae97af-399c-4eb0-b3a2-91560174d915 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:48.331382 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:48.368409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae97af-399c-4eb0-b3a2-91560174d915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:30:48.369872 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae97af-399c-4eb0-b3a2-91560174d915 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:30:48.387590 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 735/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:30:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:30:50.608751 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67c0cd-1cf3-4fc9-8b87-227dcd7702f0 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:50.609379 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67c0cd-1cf3-4fc9-8b87-227dcd7702f0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:50.609379 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67c0cd-1cf3-4fc9-8b87-227dcd7702f0 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:50.610698 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 750/1485] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:30:50.616172 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf56ee-9a17-4f47-88a8-3d96c18e0d5c None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:50.616588 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf56ee-9a17-4f47-88a8-3d96c18e0d5c None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:50.616723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf56ee-9a17-4f47-88a8-3d96c18e0d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:50.682021 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-d3bf56ee-9a17-4f47-88a8-3d96c18e0d5c None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:30:50.683237 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d3bf56ee-9a17-4f47-88a8-3d96c18e0d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:50.777853 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 736/1486] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:30:51.071236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67c0cd-1cf3-4fc9-8b87-227dcd7702f0 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:30:51.072768 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bb67c0cd-1cf3-4fc9-8b87-227dcd7702f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:51.175346 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67c0cd-1cf3-4fc9-8b87-227dcd7702f0 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:30:51.245666 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:51.247108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec426e28-db9c-4001-a6a3-1de1639df069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:30:51.247756 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec426e28-db9c-4001-a6a3-1de1639df069 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:51.248041 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec426e28-db9c-4001-a6a3-1de1639df069 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:51.248311 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec426e28-db9c-4001-a6a3-1de1639df069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:51.250522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ec426e28-db9c-4001-a6a3-1de1639df069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:51.357005 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec426e28-db9c-4001-a6a3-1de1639df069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:30:51.358422 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec426e28-db9c-4001-a6a3-1de1639df069 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:30:51.376494 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 751/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:30:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:30:53.662339 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf97ef-dd78-414f-8564-127583913c83 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:53.662910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf97ef-dd78-414f-8564-127583913c83 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:53.662910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf97ef-dd78-414f-8564-127583913c83 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:53.664230 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 737/1488] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:30:53.669412 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda4b59-fa71-4c2a-99de-8532e03db0c9 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:53.669841 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda4b59-fa71-4c2a-99de-8532e03db0c9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:53.669960 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda4b59-fa71-4c2a-99de-8532e03db0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:53.736402 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7cda4b59-fa71-4c2a-99de-8532e03db0c9 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:30:53.741162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7cda4b59-fa71-4c2a-99de-8532e03db0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:53.790704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:53.835674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 752/1489] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:30:54.125700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf97ef-dd78-414f-8564-127583913c83 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:30:54.127249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-eabf97ef-dd78-414f-8564-127583913c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:54.162649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:54.229786 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf97ef-dd78-414f-8564-127583913c83 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:30:54.304702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e08a7-2580-4919-9101-5d8b82f08e6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:30:54.305334 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e08a7-2580-4919-9101-5d8b82f08e6c service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:54.305610 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e08a7-2580-4919-9101-5d8b82f08e6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:54.305879 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e08a7-2580-4919-9101-5d8b82f08e6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:54.307493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-1b7e08a7-2580-4919-9101-5d8b82f08e6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:54.385218 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:54.413279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e08a7-2580-4919-9101-5d8b82f08e6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:30:54.414711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e08a7-2580-4919-9101-5d8b82f08e6c service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:30:54.432852 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 738/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:30:57.024559 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b52507-9e98-47de-9109-1932e520994f None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:57.025149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b52507-9e98-47de-9109-1932e520994f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:57.025149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b52507-9e98-47de-9109-1932e520994f None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:57.026752 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 753/1491] 10.222.0.42 () {54 vars in 927 bytes} [Mon Sep 19 10:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:30:57.032132 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce4b2f-11e4-49cb-a6b3-dd76fe3e9c8a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:57.032589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce4b2f-11e4-49cb-a6b3-dd76fe3e9c8a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:57.032710 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce4b2f-11e4-49cb-a6b3-dd76fe3e9c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:57.099098 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-95ce4b2f-11e4-49cb-a6b3-dd76fe3e9c8a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:30:57.100244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-95ce4b2f-11e4-49cb-a6b3-dd76fe3e9c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:57.197736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 739/1492] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Sep 19 10:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:30:57.490002 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b52507-9e98-47de-9109-1932e520994f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:30:57.491578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a5b52507-9e98-47de-9109-1932e520994f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:57.594693 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b52507-9e98-47de-9109-1932e520994f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:30:57.668187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963867e-e25c-4103-b993-0388c3b2facd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:30:57.668814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963867e-e25c-4103-b993-0388c3b2facd service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:30:57.669116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963867e-e25c-4103-b993-0388c3b2facd service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:30:57.669397 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963867e-e25c-4103-b993-0388c3b2facd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:30:57.671541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8963867e-e25c-4103-b993-0388c3b2facd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:30:57.679966 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:30:57.778856 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963867e-e25c-4103-b993-0388c3b2facd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:30:57.780807 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963867e-e25c-4103-b993-0388c3b2facd service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:30:57.798113 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 754/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:31:07.195994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=82 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:31:07.795865 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=50 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:32:49.654143 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0daf7d-f289-4c69-bd2e-f2a43fece333 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:32:49.654712 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0daf7d-f289-4c69-bd2e-f2a43fece333 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:32:49.654712 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0daf7d-f289-4c69-bd2e-f2a43fece333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:32:49.735370 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-be0daf7d-f289-4c69-bd2e-f2a43fece333 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:32:49.736578 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-be0daf7d-f289-4c69-bd2e-f2a43fece333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:32:49.845028 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 740/1494] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:32:49.852848 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488c9d9-8066-495a-a3ee-ef94297dabb7 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:32:49.853367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488c9d9-8066-495a-a3ee-ef94297dabb7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:32:49.853483 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488c9d9-8066-495a-a3ee-ef94297dabb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:32:49.873505 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7488c9d9-8066-495a-a3ee-ef94297dabb7 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:32:49.874685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7488c9d9-8066-495a-a3ee-ef94297dabb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:32:49.882915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 755/1495] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:32:49.887443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0daf7d-f289-4c69-bd2e-f2a43fece333 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:32:49.897582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0daf7d-f289-4c69-bd2e-f2a43fece333 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:32:49.902908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab4af0-f921-46b2-91a8-1667ecbb2009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:32:49.903533 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab4af0-f921-46b2-91a8-1667ecbb2009 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:32:49.903813 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab4af0-f921-46b2-91a8-1667ecbb2009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:32:49.904086 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab4af0-f921-46b2-91a8-1667ecbb2009 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:32:49.904785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab4af0-f921-46b2-91a8-1667ecbb2009 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:32:49.905031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab4af0-f921-46b2-91a8-1667ecbb2009 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:32:49.906068 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: WARNING py.warnings [None req-d7ab4af0-f921-46b2-91a8-1667ecbb2009 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 10:32:49.906068 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: warnings.warn(msg) Sep 19 10:32:49.906068 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: Sep 19 10:32:49.906955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab4af0-f921-46b2-91a8-1667ecbb2009 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:32:49.918836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 741/1496] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:32:49.926817 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488c9d9-8066-495a-a3ee-ef94297dabb7 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:32:49.935670 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488c9d9-8066-495a-a3ee-ef94297dabb7 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:32:49.940130 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27080c49-77ee-44d7-8baf-ac5e1dc920c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:32:49.940715 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27080c49-77ee-44d7-8baf-ac5e1dc920c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:32:49.940995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27080c49-77ee-44d7-8baf-ac5e1dc920c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:32:49.941296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27080c49-77ee-44d7-8baf-ac5e1dc920c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:32:49.941945 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27080c49-77ee-44d7-8baf-ac5e1dc920c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:32:49.943654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: WARNING py.warnings [None req-27080c49-77ee-44d7-8baf-ac5e1dc920c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 10:32:49.943654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: warnings.warn(msg) Sep 19 10:32:49.943654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: Sep 19 10:32:49.944846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27080c49-77ee-44d7-8baf-ac5e1dc920c5 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:32:50.102276 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 756/1497] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:32:49 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:32:50.107215 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab4af0-f921-46b2-91a8-1667ecbb2009 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:32:50.117419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab4af0-f921-46b2-91a8-1667ecbb2009 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:32:50.167310 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:32:50.173908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558003eb-e24c-4fa1-827f-81eb038bf97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:32:50.179482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558003eb-e24c-4fa1-827f-81eb038bf97f admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:32:50.179482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558003eb-e24c-4fa1-827f-81eb038bf97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:32:50.179482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558003eb-e24c-4fa1-827f-81eb038bf97f admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:32:50.179482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558003eb-e24c-4fa1-827f-81eb038bf97f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:32:50.179482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: WARNING py.warnings [None req-558003eb-e24c-4fa1-827f-81eb038bf97f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 10:32:50.179482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: warnings.warn(msg) Sep 19 10:32:50.179482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: Sep 19 10:32:50.180077 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558003eb-e24c-4fa1-827f-81eb038bf97f admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:32:50.272298 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 742/1498] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:32:50 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:32:50.277367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27080c49-77ee-44d7-8baf-ac5e1dc920c5 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:32:50.287583 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27080c49-77ee-44d7-8baf-ac5e1dc920c5 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:32:50.292235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a150dc-f90e-4c73-bac4-63a845216f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:32:50.292834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a150dc-f90e-4c73-bac4-63a845216f2b admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:32:50.293187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a150dc-f90e-4c73-bac4-63a845216f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:32:50.293522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a150dc-f90e-4c73-bac4-63a845216f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:32:50.294172 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a150dc-f90e-4c73-bac4-63a845216f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:32:50.295171 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: WARNING py.warnings [None req-d4a150dc-f90e-4c73-bac4-63a845216f2b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 10:32:50.295171 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: warnings.warn(msg) Sep 19 10:32:50.295171 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: Sep 19 10:32:50.296064 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a150dc-f90e-4c73-bac4-63a845216f2b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:32:50.308684 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 757/1499] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:32:50.312715 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558003eb-e24c-4fa1-827f-81eb038bf97f admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:32:50.323279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558003eb-e24c-4fa1-827f-81eb038bf97f admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:32:50.327881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ee844-b0cc-491b-a6fd-fda97d05d6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:32:50.328473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ee844-b0cc-491b-a6fd-fda97d05d6ae admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:32:50.328752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ee844-b0cc-491b-a6fd-fda97d05d6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:32:50.329026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ee844-b0cc-491b-a6fd-fda97d05d6ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:32:50.329690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ee844-b0cc-491b-a6fd-fda97d05d6ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:32:50.330689 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: WARNING py.warnings [None req-482ee844-b0cc-491b-a6fd-fda97d05d6ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 10:32:50.330689 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: warnings.warn(msg) Sep 19 10:32:50.330689 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: Sep 19 10:32:50.331577 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ee844-b0cc-491b-a6fd-fda97d05d6ae admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:32:50.345400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 743/1500] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:32:50.349444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a150dc-f90e-4c73-bac4-63a845216f2b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:32:50.359577 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a150dc-f90e-4c73-bac4-63a845216f2b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:32:50.364227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a0183-ec74-4cb4-8c77-f25d92e7e648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:32:50.364871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a0183-ec74-4cb4-8c77-f25d92e7e648 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:32:50.365185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a0183-ec74-4cb4-8c77-f25d92e7e648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:32:50.365461 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a0183-ec74-4cb4-8c77-f25d92e7e648 admin admin] PATH_INFO: `/v3/projects/ea1912b38d2e472b86be6fd60a5697b8/users/28f9deef5ec347669be6fa4b47dda5be/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:32:50.380961 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a0183-ec74-4cb4-8c77-f25d92e7e648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea1912b38d2e472b86be6fd60a5697b8, user_id=28f9deef5ec347669be6fa4b47dda5be, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:32:50.382002 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: WARNING py.warnings [None req-8b5a0183-ec74-4cb4-8c77-f25d92e7e648 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 10:32:50.382002 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: warnings.warn(msg) Sep 19 10:32:50.382002 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: Sep 19 10:32:50.383968 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a0183-ec74-4cb4-8c77-f25d92e7e648 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:32:50.478199 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 758/1501] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:32:50 2022] PUT /identity/v3/projects/ea1912b38d2e472b86be6fd60a5697b8/users/28f9deef5ec347669be6fa4b47dda5be/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:32:50.490549 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5d9b9-42c5-4675-9b54-60ec6503d737 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:32:50.490841 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5d9b9-42c5-4675-9b54-60ec6503d737 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:32:50.491121 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5d9b9-42c5-4675-9b54-60ec6503d737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:32:50.519917 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-51e5d9b9-42c5-4675-9b54-60ec6503d737 None None] MFA Rules not processed for user `28f9deef5ec347669be6fa4b47dda5be`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:32:50.521152 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-51e5d9b9-42c5-4675-9b54-60ec6503d737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:32:50.585348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 744/1502] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:32:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:32:50.598408 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69481fdb-802d-4282-affc-f6c2193a6d1e None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:32:50.598690 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69481fdb-802d-4282-affc-f6c2193a6d1e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:32:50.598960 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69481fdb-802d-4282-affc-f6c2193a6d1e None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:32:50.600285 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 759/1503] 10.222.0.42 () {54 vars in 962 bytes} [Mon Sep 19 10:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:32:50.605347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43463444-7891-48c1-b1b1-a301984953d0 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:32:50.605632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43463444-7891-48c1-b1b1-a301984953d0 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:32:50.608795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43463444-7891-48c1-b1b1-a301984953d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:32:50.782146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-43463444-7891-48c1-b1b1-a301984953d0 None None] MFA Rules not processed for user `e8a10f3332584a90a2231564527078cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:32:50.783246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-43463444-7891-48c1-b1b1-a301984953d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:32:50.858729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 745/1504] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Sep 19 10:32:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:32:50.866461 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69481fdb-802d-4282-affc-f6c2193a6d1e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:32:50.877396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69481fdb-802d-4282-affc-f6c2193a6d1e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:32:50.880805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b41c4-972c-4caa-902b-2d62b9ed194f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:32:50.881438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b41c4-972c-4caa-902b-2d62b9ed194f service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:32:50.881697 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b41c4-972c-4caa-902b-2d62b9ed194f service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:32:50.881967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b41c4-972c-4caa-902b-2d62b9ed194f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:32:50.890054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b41c4-972c-4caa-902b-2d62b9ed194f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:32:50.891494 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b41c4-972c-4caa-902b-2d62b9ed194f service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:32:50.900578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 760/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:32:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:32:50.947883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c53ab7-3f45-4289-8391-555c130d6838 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:32:50.948185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c53ab7-3f45-4289-8391-555c130d6838 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:32:50.948453 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c53ab7-3f45-4289-8391-555c130d6838 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:32:50.949726 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 746/1506] 10.222.0.42 () {52 vars in 830 bytes} [Mon Sep 19 10:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:32:50.953341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b41c4-972c-4caa-902b-2d62b9ed194f service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:32:51.014345 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b41c4-972c-4caa-902b-2d62b9ed194f service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:32:51.018967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839855e7-bc72-4c4c-874c-0a84ab95b8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:32:51.019588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839855e7-bc72-4c4c-874c-0a84ab95b8d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:32:51.019871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839855e7-bc72-4c4c-874c-0a84ab95b8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:32:51.020148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839855e7-bc72-4c4c-874c-0a84ab95b8d4 admin admin] PATH_INFO: `/v3/users/28f9deef5ec347669be6fa4b47dda5be` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:32:51.021294 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839855e7-bc72-4c4c-874c-0a84ab95b8d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28f9deef5ec347669be6fa4b47dda5be)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:32:51.022291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: WARNING py.warnings [None req-839855e7-bc72-4c4c-874c-0a84ab95b8d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 10:32:51.022291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: warnings.warn(msg) Sep 19 10:32:51.022291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: Sep 19 10:32:51.023758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839855e7-bc72-4c4c-874c-0a84ab95b8d4 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:32:51.248777 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-839855e7-bc72-4c4c-874c-0a84ab95b8d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28f9deef5ec347669be6fa4b47dda5be'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:32:51.249628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-839855e7-bc72-4c4c-874c-0a84ab95b8d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28f9deef5ec347669be6fa4b47dda5be'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:32:51.335304 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-839855e7-bc72-4c4c-874c-0a84ab95b8d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28f9deef5ec347669be6fa4b47dda5be'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:32:51.335932 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-839855e7-bc72-4c4c-874c-0a84ab95b8d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28f9deef5ec347669be6fa4b47dda5be'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:32:51.361629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-839855e7-bc72-4c4c-874c-0a84ab95b8d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28f9deef5ec347669be6fa4b47dda5be'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:32:51.371679 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:32:51.381356 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 761/1507] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:32:50 2022] DELETE /identity/v3/users/28f9deef5ec347669be6fa4b47dda5be => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:32:51.385836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732f4bc-1470-409a-9c81-be0893c6875e None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:32:51.386147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732f4bc-1470-409a-9c81-be0893c6875e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:32:51.386422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732f4bc-1470-409a-9c81-be0893c6875e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:32:51.405660 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-b732f4bc-1470-409a-9c81-be0893c6875e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:32:51.406854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b732f4bc-1470-409a-9c81-be0893c6875e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:32:51.415743 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:32:51.463751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 747/1508] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:32:51.473594 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839855e7-bc72-4c4c-874c-0a84ab95b8d4 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:32:51.475049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-839855e7-bc72-4c4c-874c-0a84ab95b8d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:32:51.552230 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839855e7-bc72-4c4c-874c-0a84ab95b8d4 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:32:51.555775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686acac-d592-455e-9925-e2c6bccf8bf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:32:51.556383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686acac-d592-455e-9925-e2c6bccf8bf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:32:51.556677 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686acac-d592-455e-9925-e2c6bccf8bf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:32:51.556947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686acac-d592-455e-9925-e2c6bccf8bf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:32:51.565732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686acac-d592-455e-9925-e2c6bccf8bf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:32:51.567144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686acac-d592-455e-9925-e2c6bccf8bf1 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:32:51.576154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 762/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:32:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:32:52.094014 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732f4bc-1470-409a-9c81-be0893c6875e None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:32:52.095483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b732f4bc-1470-409a-9c81-be0893c6875e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:32:52.107275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732f4bc-1470-409a-9c81-be0893c6875e None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:32:52.111731 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a824dbb-b2c9-4765-9b28-5f2a30143963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:32:52.112369 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a824dbb-b2c9-4765-9b28-5f2a30143963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:32:52.112646 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a824dbb-b2c9-4765-9b28-5f2a30143963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:32:52.112926 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a824dbb-b2c9-4765-9b28-5f2a30143963 admin admin] PATH_INFO: `/v3/projects/ea1912b38d2e472b86be6fd60a5697b8` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:32:52.114085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a824dbb-b2c9-4765-9b28-5f2a30143963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea1912b38d2e472b86be6fd60a5697b8)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:32:52.115108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: WARNING py.warnings [None req-4a824dbb-b2c9-4765-9b28-5f2a30143963 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 10:32:52.115108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: warnings.warn(msg) Sep 19 10:32:52.115108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: Sep 19 10:32:52.116307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a824dbb-b2c9-4765-9b28-5f2a30143963 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:32:52.223016 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-4a824dbb-b2c9-4765-9b28-5f2a30143963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea1912b38d2e472b86be6fd60a5697b8', 'request_id': 'req-4a824dbb-b2c9-4765-9b28-5f2a30143963'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:32:52.229015 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-4a824dbb-b2c9-4765-9b28-5f2a30143963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea1912b38d2e472b86be6fd60a5697b8', 'request_id': 'req-4a824dbb-b2c9-4765-9b28-5f2a30143963'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:32:52.241556 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-4a824dbb-b2c9-4765-9b28-5f2a30143963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea1912b38d2e472b86be6fd60a5697b8', 'request_id': 'req-4a824dbb-b2c9-4765-9b28-5f2a30143963'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:32:52.331121 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-4a824dbb-b2c9-4765-9b28-5f2a30143963 admin admin] The token cache is being invalidate because project ea1912b38d2e472b86be6fd60a5697b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:32:52.331406 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-4a824dbb-b2c9-4765-9b28-5f2a30143963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:32:52.332101 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-4a824dbb-b2c9-4765-9b28-5f2a30143963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:32:52.333605 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 748/1510] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:32:52 2022] DELETE /identity/v3/projects/ea1912b38d2e472b86be6fd60a5697b8 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:33:01.572174 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=38 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:33:02.327817 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:34:19.735787 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b68a0-28f2-44f7-9ad7-362deccb5c09 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:19.736397 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b68a0-28f2-44f7-9ad7-362deccb5c09 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:19.736397 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b68a0-28f2-44f7-9ad7-362deccb5c09 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:19.737764 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 10:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:34:19.742672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6719f4-17d4-4c38-b3da-f40999ca8206 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:19.743128 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6719f4-17d4-4c38-b3da-f40999ca8206 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:19.743239 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6719f4-17d4-4c38-b3da-f40999ca8206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:19.764325 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-9f6719f4-17d4-4c38-b3da-f40999ca8206 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:34:19.765670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-9f6719f4-17d4-4c38-b3da-f40999ca8206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:34:19.823695 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 10:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:34:19.977842 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b68a0-28f2-44f7-9ad7-362deccb5c09 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:34:19.979585 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7f0b68a0-28f2-44f7-9ad7-362deccb5c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:34:20.080175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b68a0-28f2-44f7-9ad7-362deccb5c09 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:34:20.086406 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa957e-01c3-4ff3-a52e-cebb585f24bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:34:20.087011 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa957e-01c3-4ff3-a52e-cebb585f24bd service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:20.087292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa957e-01c3-4ff3-a52e-cebb585f24bd service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:20.087566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa957e-01c3-4ff3-a52e-cebb585f24bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:20.095602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa957e-01c3-4ff3-a52e-cebb585f24bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:34:20.097202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa957e-01c3-4ff3-a52e-cebb585f24bd service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:34:20.106459 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 764/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:34:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:34:21.178443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42426c6a-e277-4875-80c9-9b8118e9260f None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:21.178999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42426c6a-e277-4875-80c9-9b8118e9260f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:21.178999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42426c6a-e277-4875-80c9-9b8118e9260f None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:21.180380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 10:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:34:21.186541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a87e8c-067f-4aa4-a1aa-15fa49c50af6 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:21.186998 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a87e8c-067f-4aa4-a1aa-15fa49c50af6 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:21.187097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a87e8c-067f-4aa4-a1aa-15fa49c50af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:21.206383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-94a87e8c-067f-4aa4-a1aa-15fa49c50af6 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:34:21.207600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-94a87e8c-067f-4aa4-a1aa-15fa49c50af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:34:21.215960 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 10:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:34:21.367290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42426c6a-e277-4875-80c9-9b8118e9260f None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:34:21.382295 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42426c6a-e277-4875-80c9-9b8118e9260f None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:34:21.387749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503e5a5-bdf5-4b2e-aac8-ef77d051fe2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:34:21.388350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503e5a5-bdf5-4b2e-aac8-ef77d051fe2c service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:21.388629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503e5a5-bdf5-4b2e-aac8-ef77d051fe2c service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:21.388905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503e5a5-bdf5-4b2e-aac8-ef77d051fe2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:21.397933 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503e5a5-bdf5-4b2e-aac8-ef77d051fe2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:34:21.399517 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503e5a5-bdf5-4b2e-aac8-ef77d051fe2c service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:34:21.410435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 751/1516] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:34:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:34:21.492579 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fb7af-8756-4062-8db1-d426f586bb4e None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:21.492870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fb7af-8756-4062-8db1-d426f586bb4e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:21.493180 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fb7af-8756-4062-8db1-d426f586bb4e None None] PATH_INFO: `/v3` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:21.494516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 766/1517] 10.222.0.73 () {54 vars in 925 bytes} [Mon Sep 19 10:34:21 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:34:21.508261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35033331-e02f-4a30-b711-ec5e25dd6d9a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:21.509730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35033331-e02f-4a30-b711-ec5e25dd6d9a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:21.510006 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35033331-e02f-4a30-b711-ec5e25dd6d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:21.560580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-35033331-e02f-4a30-b711-ec5e25dd6d9a None None] MFA Rules not processed for user `ce0ef47a60f54ca1853fd267e308c58b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:34:21.561808 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-35033331-e02f-4a30-b711-ec5e25dd6d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:34:21.631713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 752/1518] 10.222.0.73 () {58 vars in 1015 bytes} [Mon Sep 19 10:34:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:34:30.660657 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac967098-34d9-438d-a1ee-f2acf2fbb0d5 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:30.661184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac967098-34d9-438d-a1ee-f2acf2fbb0d5 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:30.661291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac967098-34d9-438d-a1ee-f2acf2fbb0d5 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:30.662569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 767/1519] 10.222.0.42 () {54 vars in 961 bytes} [Mon Sep 19 10:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:34:30.667694 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b83625-9457-42f7-95b9-dfaa49104a54 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:30.668150 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b83625-9457-42f7-95b9-dfaa49104a54 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:30.668264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b83625-9457-42f7-95b9-dfaa49104a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:30.687868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-54b83625-9457-42f7-95b9-dfaa49104a54 None None] MFA Rules not processed for user `ce0ef47a60f54ca1853fd267e308c58b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:34:30.689072 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-54b83625-9457-42f7-95b9-dfaa49104a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:34:30.698099 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 753/1520] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Sep 19 10:34:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:34:30.706092 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac967098-34d9-438d-a1ee-f2acf2fbb0d5 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:34:30.719699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac967098-34d9-438d-a1ee-f2acf2fbb0d5 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:34:30.724950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce004c66-1192-46c7-8a5a-878fb9bf70c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0ef47a60f54ca1853fd267e308c58b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:34:30.725568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce004c66-1192-46c7-8a5a-878fb9bf70c7 service placement] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:30.725847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce004c66-1192-46c7-8a5a-878fb9bf70c7 service placement] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:30.726122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce004c66-1192-46c7-8a5a-878fb9bf70c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:30.734160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce004c66-1192-46c7-8a5a-878fb9bf70c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:34:30.735713 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce004c66-1192-46c7-8a5a-878fb9bf70c7 service placement] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:34:30.745637 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 768/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:34:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:34:32.892749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefb868-07d2-4e3b-b70c-1a76fe2e9876 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:32.893286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefb868-07d2-4e3b-b70c-1a76fe2e9876 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:32.893286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefb868-07d2-4e3b-b70c-1a76fe2e9876 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:32.894635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 10:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:34:32.901561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426ba03-65d9-44c8-92bc-a285061a5352 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:32.902029 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426ba03-65d9-44c8-92bc-a285061a5352 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:32.902127 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426ba03-65d9-44c8-92bc-a285061a5352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:32.921745 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f426ba03-65d9-44c8-92bc-a285061a5352 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:34:32.922941 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f426ba03-65d9-44c8-92bc-a285061a5352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:34:32.930840 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 10:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:34:33.084521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefb868-07d2-4e3b-b70c-1a76fe2e9876 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:34:33.099299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefb868-07d2-4e3b-b70c-1a76fe2e9876 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:34:33.104819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1cfb4-b6df-40a9-8e92-ff056571b533 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:34:33.105446 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1cfb4-b6df-40a9-8e92-ff056571b533 service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:33.105728 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1cfb4-b6df-40a9-8e92-ff056571b533 service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:33.106006 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1cfb4-b6df-40a9-8e92-ff056571b533 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:33.114090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1cfb4-b6df-40a9-8e92-ff056571b533 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:34:33.119518 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1cfb4-b6df-40a9-8e92-ff056571b533 service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:34:33.125004 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 755/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:34:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:34:34.200852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419ddda-1f3f-4b51-a4a5-46d318d4eb23 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:34.201444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419ddda-1f3f-4b51-a4a5-46d318d4eb23 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:34.201444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419ddda-1f3f-4b51-a4a5-46d318d4eb23 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:34.202788 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 10:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:34:34.208660 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83454b-3d9d-44ca-9c86-6d79ee2af8b0 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:34.209144 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83454b-3d9d-44ca-9c86-6d79ee2af8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:34.209256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83454b-3d9d-44ca-9c86-6d79ee2af8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:34.228858 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-bd83454b-3d9d-44ca-9c86-6d79ee2af8b0 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:34:34.230140 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-bd83454b-3d9d-44ca-9c86-6d79ee2af8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:34:34.237958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 10:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:34:34.390680 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419ddda-1f3f-4b51-a4a5-46d318d4eb23 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:34:34.405330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419ddda-1f3f-4b51-a4a5-46d318d4eb23 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:34:34.411225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07d8f2-257e-4025-8cad-f90b21a9d377 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:34:34.411812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07d8f2-257e-4025-8cad-f90b21a9d377 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:34.412086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07d8f2-257e-4025-8cad-f90b21a9d377 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:34.412351 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07d8f2-257e-4025-8cad-f90b21a9d377 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:34.420293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07d8f2-257e-4025-8cad-f90b21a9d377 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:34:34.421935 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07d8f2-257e-4025-8cad-f90b21a9d377 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:34:34.431324 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 771/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:34:38.661368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e222fd8-4876-476b-a23e-5d9925872bbc None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:38.661921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e222fd8-4876-476b-a23e-5d9925872bbc None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:38.661921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e222fd8-4876-476b-a23e-5d9925872bbc None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:38.663243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 10:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:34:38.668425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593db113-009f-4746-93ea-2944dbc8fc23 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:38.668881 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593db113-009f-4746-93ea-2944dbc8fc23 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:38.668999 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593db113-009f-4746-93ea-2944dbc8fc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:38.690706 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-593db113-009f-4746-93ea-2944dbc8fc23 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:34:38.692082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-593db113-009f-4746-93ea-2944dbc8fc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:34:38.699959 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 10:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:34:38.987250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e222fd8-4876-476b-a23e-5d9925872bbc None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:34:38.989467 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4e222fd8-4876-476b-a23e-5d9925872bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:34:39.017947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:34:39.050105 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e222fd8-4876-476b-a23e-5d9925872bbc None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:34:39.053699 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713b4e1-c487-4b1d-9572-318d08930108 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:34:39.054372 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713b4e1-c487-4b1d-9572-318d08930108 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:39.054689 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713b4e1-c487-4b1d-9572-318d08930108 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:39.054963 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713b4e1-c487-4b1d-9572-318d08930108 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:39.064313 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713b4e1-c487-4b1d-9572-318d08930108 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:34:39.065771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713b4e1-c487-4b1d-9572-318d08930108 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:34:39.074995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 758/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:34:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:34:41.214998 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebb4df-e7f3-4f1e-a302-942ea98b13c2 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:41.215537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebb4df-e7f3-4f1e-a302-942ea98b13c2 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:41.215537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebb4df-e7f3-4f1e-a302-942ea98b13c2 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:41.216954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 10:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:34:41.222259 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7419e8-3b0d-4034-8b4a-3a2798da83a7 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:41.223251 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7419e8-3b0d-4034-8b4a-3a2798da83a7 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:41.223434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7419e8-3b0d-4034-8b4a-3a2798da83a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:41.242596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-8f7419e8-3b0d-4034-8b4a-3a2798da83a7 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:34:41.243775 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-8f7419e8-3b0d-4034-8b4a-3a2798da83a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:34:41.251604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 10:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:34:41.540563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebb4df-e7f3-4f1e-a302-942ea98b13c2 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:34:41.552574 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebb4df-e7f3-4f1e-a302-942ea98b13c2 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:34:41.556127 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e34db-97a0-44ce-8c58-6bbb2a2db1da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:34:41.556743 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e34db-97a0-44ce-8c58-6bbb2a2db1da service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:41.557026 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e34db-97a0-44ce-8c58-6bbb2a2db1da service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:41.557358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e34db-97a0-44ce-8c58-6bbb2a2db1da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:41.567000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e34db-97a0-44ce-8c58-6bbb2a2db1da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:34:41.568417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e34db-97a0-44ce-8c58-6bbb2a2db1da service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:34:41.578417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 774/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:34:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:34:45.362045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912cb1f-b318-48f7-9801-0b6ac23a31f3 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:45.362617 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912cb1f-b318-48f7-9801-0b6ac23a31f3 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:45.362617 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912cb1f-b318-48f7-9801-0b6ac23a31f3 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:45.363949 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 10:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:34:45.369231 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264321b-4723-475e-9c89-9e4b16f3cd09 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:45.369662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264321b-4723-475e-9c89-9e4b16f3cd09 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:45.369790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264321b-4723-475e-9c89-9e4b16f3cd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:45.377490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=74 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:34:45.389887 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a264321b-4723-475e-9c89-9e4b16f3cd09 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:34:45.391099 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a264321b-4723-475e-9c89-9e4b16f3cd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:34:45.399688 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 10:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:34:45.708779 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912cb1f-b318-48f7-9801-0b6ac23a31f3 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:34:45.710470 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-8912cb1f-b318-48f7-9801-0b6ac23a31f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:34:45.766205 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912cb1f-b318-48f7-9801-0b6ac23a31f3 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:34:45.769770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e67b0e-64bb-4e6f-9f06-dce1726993e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:34:45.770615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e67b0e-64bb-4e6f-9f06-dce1726993e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:45.770893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e67b0e-64bb-4e6f-9f06-dce1726993e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:45.771162 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e67b0e-64bb-4e6f-9f06-dce1726993e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:45.780055 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e67b0e-64bb-4e6f-9f06-dce1726993e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:34:45.781509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e67b0e-64bb-4e6f-9f06-dce1726993e0 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:34:45.790596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 761/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:34:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:34:47.918070 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27ff40-ee73-49e7-aeb1-d5f138a6e313 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:47.918632 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27ff40-ee73-49e7-aeb1-d5f138a6e313 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:47.918632 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27ff40-ee73-49e7-aeb1-d5f138a6e313 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:47.919997 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 10:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:34:47.925276 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62574ef-1502-4587-a989-fa23fa0300a9 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:47.925683 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62574ef-1502-4587-a989-fa23fa0300a9 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:47.925835 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62574ef-1502-4587-a989-fa23fa0300a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:47.944747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-c62574ef-1502-4587-a989-fa23fa0300a9 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:34:47.945969 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c62574ef-1502-4587-a989-fa23fa0300a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:34:47.953728 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 10:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:34:48.262408 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27ff40-ee73-49e7-aeb1-d5f138a6e313 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:34:48.264098 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cb27ff40-ee73-49e7-aeb1-d5f138a6e313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:34:48.277783 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27ff40-ee73-49e7-aeb1-d5f138a6e313 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:34:48.281297 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330884e9-a404-43be-a25a-24a9a318d655 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:34:48.281977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330884e9-a404-43be-a25a-24a9a318d655 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:34:48.282260 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330884e9-a404-43be-a25a-24a9a318d655 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:34:48.282534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330884e9-a404-43be-a25a-24a9a318d655 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:34:48.291294 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330884e9-a404-43be-a25a-24a9a318d655 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:34:48.292700 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330884e9-a404-43be-a25a-24a9a318d655 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:34:48.301974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 777/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:34:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:34:57.939933 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=47 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:34:58.297872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=8 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:14.237296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cccbaf-a8ca-4933-9edb-917a644f6f88 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:14.237944 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cccbaf-a8ca-4933-9edb-917a644f6f88 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:14.237944 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cccbaf-a8ca-4933-9edb-917a644f6f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:14.262860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-e8cccbaf-a8ca-4933-9edb-917a644f6f88 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:14.264656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e8cccbaf-a8ca-4933-9edb-917a644f6f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:14.277375 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 763/1540] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:14.287619 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54eec96-7ebc-4a8a-8114-217c55b756bc None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:14.288238 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54eec96-7ebc-4a8a-8114-217c55b756bc None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:14.288238 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54eec96-7ebc-4a8a-8114-217c55b756bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:14.316290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a54eec96-7ebc-4a8a-8114-217c55b756bc None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:14.317502 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a54eec96-7ebc-4a8a-8114-217c55b756bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:14.348891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 778/1541] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:14.353799 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cccbaf-a8ca-4933-9edb-917a644f6f88 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:14.370735 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cccbaf-a8ca-4933-9edb-917a644f6f88 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:14.376553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d9b2f-d346-47e0-b217-9fc5a485ea7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:14.377392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d9b2f-d346-47e0-b217-9fc5a485ea7b admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:14.377675 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d9b2f-d346-47e0-b217-9fc5a485ea7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:14.377950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d9b2f-d346-47e0-b217-9fc5a485ea7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:14.378608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d9b2f-d346-47e0-b217-9fc5a485ea7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:36:14.378911 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d9b2f-d346-47e0-b217-9fc5a485ea7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:14.380516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d9b2f-d346-47e0-b217-9fc5a485ea7b admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:14.398460 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 764/1542] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:36:14.407993 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54eec96-7ebc-4a8a-8114-217c55b756bc None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:14.417158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54eec96-7ebc-4a8a-8114-217c55b756bc None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:14.425289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cede4-2b8a-44ca-9123-ddd1f7790084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:14.425915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cede4-2b8a-44ca-9123-ddd1f7790084 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:14.426200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cede4-2b8a-44ca-9123-ddd1f7790084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:14.426597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cede4-2b8a-44ca-9123-ddd1f7790084 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:14.427117 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443cede4-2b8a-44ca-9123-ddd1f7790084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:14.429027 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443cede4-2b8a-44ca-9123-ddd1f7790084 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:14.547583 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff91eb6-6aef-426f-b636-4801aa564a88 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:14.557376 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff91eb6-6aef-426f-b636-4801aa564a88 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:14.557376 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff91eb6-6aef-426f-b636-4801aa564a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:14.592231 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 779/1543] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:14 2022] POST /identity/v3/projects => generated 381 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:14.595010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-eff91eb6-6aef-426f-b636-4801aa564a88 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:14.596272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cede4-2b8a-44ca-9123-ddd1f7790084 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:14.604436 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-eff91eb6-6aef-426f-b636-4801aa564a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:14.702487 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cede4-2b8a-44ca-9123-ddd1f7790084 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:14.713848 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e957710-f759-4d05-b82a-7973c0466f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:14.715258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e957710-f759-4d05-b82a-7973c0466f5c admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:14.716260 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e957710-f759-4d05-b82a-7973c0466f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:14.720693 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e957710-f759-4d05-b82a-7973c0466f5c admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:14.722081 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e957710-f759-4d05-b82a-7973c0466f5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:14.723653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: WARNING py.warnings [None req-7e957710-f759-4d05-b82a-7973c0466f5c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 10:36:14.723653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: warnings.warn(msg) Sep 19 10:36:14.723653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: Sep 19 10:36:14.726555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e957710-f759-4d05-b82a-7973c0466f5c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:14.734240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 765/1544] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:14.735865 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c59d66-56ff-487f-bce6-29a7d0a40296 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:14.736396 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c59d66-56ff-487f-bce6-29a7d0a40296 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:14.736900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c59d66-56ff-487f-bce6-29a7d0a40296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:14.759269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-b3c59d66-56ff-487f-bce6-29a7d0a40296 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:14.764258 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b3c59d66-56ff-487f-bce6-29a7d0a40296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:14.770619 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 766/1545] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:14.772179 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690ae7a-69de-45ad-b714-26d4a4672d15 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:14.772669 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690ae7a-69de-45ad-b714-26d4a4672d15 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:14.773337 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690ae7a-69de-45ad-b714-26d4a4672d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:14.794655 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-4690ae7a-69de-45ad-b714-26d4a4672d15 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:14.796200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4690ae7a-69de-45ad-b714-26d4a4672d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:14.805200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 767/1546] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:14.806690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b27a63-eaaa-40b0-b5b2-b594fb3b950c None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:14.807192 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b27a63-eaaa-40b0-b5b2-b594fb3b950c None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:14.807667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b27a63-eaaa-40b0-b5b2-b594fb3b950c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:14.827450 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 780/1547] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:14 2022] POST /identity/v3/users => generated 291 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:36:14.827450 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb79e4-9e4e-4493-96a3-e8a0da37a07c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:14.832160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb79e4-9e4e-4493-96a3-e8a0da37a07c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:14.832160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb79e4-9e4e-4493-96a3-e8a0da37a07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:14.840203 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-41b27a63-eaaa-40b0-b5b2-b594fb3b950c None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:14.841474 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-41b27a63-eaaa-40b0-b5b2-b594fb3b950c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:14.847918 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-45fb79e4-9e4e-4493-96a3-e8a0da37a07c None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:14.849016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-45fb79e4-9e4e-4493-96a3-e8a0da37a07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:14.849415 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 768/1548] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:14.854365 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b543d-965c-4aaf-83c1-ee14aa2821a2 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:14.854843 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b543d-965c-4aaf-83c1-ee14aa2821a2 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:14.855240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b543d-965c-4aaf-83c1-ee14aa2821a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:14.857590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 781/1549] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:14.858893 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc181b-26d5-40e8-a71c-3a752700f36c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:14.859179 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc181b-26d5-40e8-a71c-3a752700f36c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:14.859453 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc181b-26d5-40e8-a71c-3a752700f36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:14.875869 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-be2b543d-965c-4aaf-83c1-ee14aa2821a2 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:14.877094 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-be2b543d-965c-4aaf-83c1-ee14aa2821a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:14.878826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4ecc181b-26d5-40e8-a71c-3a752700f36c None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:14.880002 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4ecc181b-26d5-40e8-a71c-3a752700f36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:14.885087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 769/1550] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:14.886268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9861c3-15b7-4435-943c-44ca0a478c6e None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:14.886551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9861c3-15b7-4435-943c-44ca0a478c6e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:14.886823 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9861c3-15b7-4435-943c-44ca0a478c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:14.890208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 782/1551] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:14.891384 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1085e0-1e30-4251-ab09-ed8ae86a287e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:14.891668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1085e0-1e30-4251-ab09-ed8ae86a287e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:14.891948 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1085e0-1e30-4251-ab09-ed8ae86a287e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:14.913194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-fe9861c3-15b7-4435-943c-44ca0a478c6e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:14.913462 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-fb1085e0-1e30-4251-ab09-ed8ae86a287e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:14.914394 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-fe9861c3-15b7-4435-943c-44ca0a478c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:14.914575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-fb1085e0-1e30-4251-ab09-ed8ae86a287e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:14.922777 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 783/1552] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:14.923545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1085e0-1e30-4251-ab09-ed8ae86a287e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:14.926675 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 770/1553] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:14.927925 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff97b1a-8713-4264-87e7-7b7dacc52f9e None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:14.928208 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff97b1a-8713-4264-87e7-7b7dacc52f9e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:14.928483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff97b1a-8713-4264-87e7-7b7dacc52f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:14.935103 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1085e0-1e30-4251-ab09-ed8ae86a287e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:14.941101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9db113-7db1-443b-ae07-9524834fc59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:14.941707 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9db113-7db1-443b-ae07-9524834fc59c admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:14.941984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9db113-7db1-443b-ae07-9524834fc59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:14.942254 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9db113-7db1-443b-ae07-9524834fc59c admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:14.943319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9db113-7db1-443b-ae07-9524834fc59c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:14.944924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9db113-7db1-443b-ae07-9524834fc59c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:14.948567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-cff97b1a-8713-4264-87e7-7b7dacc52f9e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:14.949782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-cff97b1a-8713-4264-87e7-7b7dacc52f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:14.955589 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 784/1554] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:14.956872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb992d81-f309-43da-90aa-fd76f041b93a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:14.957258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb992d81-f309-43da-90aa-fd76f041b93a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:14.957538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb992d81-f309-43da-90aa-fd76f041b93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:14.959887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 771/1555] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:14.960671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff97b1a-8713-4264-87e7-7b7dacc52f9e None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:14.971921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff97b1a-8713-4264-87e7-7b7dacc52f9e None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:14.976600 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf7aa4-6f9a-43e6-9d45-3fb30043a197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:14.976758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-bb992d81-f309-43da-90aa-fd76f041b93a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:14.977235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf7aa4-6f9a-43e6-9d45-3fb30043a197 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:14.977521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf7aa4-6f9a-43e6-9d45-3fb30043a197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:14.977795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf7aa4-6f9a-43e6-9d45-3fb30043a197 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:14.977974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bb992d81-f309-43da-90aa-fd76f041b93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:14.978438 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf7aa4-6f9a-43e6-9d45-3fb30043a197 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:36:14.978810 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf7aa4-6f9a-43e6-9d45-3fb30043a197 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:14.980380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf7aa4-6f9a-43e6-9d45-3fb30043a197 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:14.986476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 785/1556] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:14.987256 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb992d81-f309-43da-90aa-fd76f041b93a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:15.000835 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 772/1557] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:36:15.001029 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb992d81-f309-43da-90aa-fd76f041b93a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:15.001163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733e5a8-c9dd-479c-8840-d41bed2a23e8 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.001163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733e5a8-c9dd-479c-8840-d41bed2a23e8 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.001163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733e5a8-c9dd-479c-8840-d41bed2a23e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.006710 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab67c0f-4c17-41b9-9b0f-969ab908fea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:15.007387 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab67c0f-4c17-41b9-9b0f-969ab908fea7 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.007671 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab67c0f-4c17-41b9-9b0f-969ab908fea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.007947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab67c0f-4c17-41b9-9b0f-969ab908fea7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.008655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab67c0f-4c17-41b9-9b0f-969ab908fea7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:36:15.008960 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab67c0f-4c17-41b9-9b0f-969ab908fea7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:15.010002 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: WARNING py.warnings [None req-aab67c0f-4c17-41b9-9b0f-969ab908fea7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 10:36:15.010002 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: warnings.warn(msg) Sep 19 10:36:15.010002 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: Sep 19 10:36:15.010881 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab67c0f-4c17-41b9-9b0f-969ab908fea7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:15.018067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-c733e5a8-c9dd-479c-8840-d41bed2a23e8 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:15.019267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c733e5a8-c9dd-479c-8840-d41bed2a23e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:15.023334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 786/1558] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:36:15.024126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab67c0f-4c17-41b9-9b0f-969ab908fea7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:15.027431 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 773/1559] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:15.028782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4ca9b-51fe-42c6-897c-f97df2f88777 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.029036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4ca9b-51fe-42c6-897c-f97df2f88777 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.029343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4ca9b-51fe-42c6-897c-f97df2f88777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.033666 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab67c0f-4c17-41b9-9b0f-969ab908fea7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:15.038369 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245188f-e492-4141-96c2-d13a9a626091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:15.038964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245188f-e492-4141-96c2-d13a9a626091 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.039249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245188f-e492-4141-96c2-d13a9a626091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.039527 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245188f-e492-4141-96c2-d13a9a626091 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.040169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245188f-e492-4141-96c2-d13a9a626091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:36:15.040476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245188f-e492-4141-96c2-d13a9a626091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:15.042092 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245188f-e492-4141-96c2-d13a9a626091 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:15.048666 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-2cf4ca9b-51fe-42c6-897c-f97df2f88777 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:15.049886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-2cf4ca9b-51fe-42c6-897c-f97df2f88777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:15.054797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 787/1560] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:36:15.055653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245188f-e492-4141-96c2-d13a9a626091 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:15.057964 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 774/1561] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:15.058996 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1603ff7-37ab-4297-8dd3-9ba649c73806 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.059279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1603ff7-37ab-4297-8dd3-9ba649c73806 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.059555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1603ff7-37ab-4297-8dd3-9ba649c73806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.069566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245188f-e492-4141-96c2-d13a9a626091 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:15.074080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d598786-0c1a-48ee-9547-789aeef7395d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:15.074670 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d598786-0c1a-48ee-9547-789aeef7395d admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.075069 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d598786-0c1a-48ee-9547-789aeef7395d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.075348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d598786-0c1a-48ee-9547-789aeef7395d admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.076044 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d598786-0c1a-48ee-9547-789aeef7395d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:15.077652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d598786-0c1a-48ee-9547-789aeef7395d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:15.079233 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-c1603ff7-37ab-4297-8dd3-9ba649c73806 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:15.080435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c1603ff7-37ab-4297-8dd3-9ba649c73806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:15.087763 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 788/1562] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:15.088672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d598786-0c1a-48ee-9547-789aeef7395d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:15.088955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 775/1563] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:15.089915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1603ff7-37ab-4297-8dd3-9ba649c73806 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:15.099305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1603ff7-37ab-4297-8dd3-9ba649c73806 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:15.100085 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d598786-0c1a-48ee-9547-789aeef7395d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:15.103906 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac913c02-8c2d-4c7e-be61-cef08bad5f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:15.104495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac913c02-8c2d-4c7e-be61-cef08bad5f03 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.104612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405a377-481c-4eae-a367-5136bdc34618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:15.104774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac913c02-8c2d-4c7e-be61-cef08bad5f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.105048 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac913c02-8c2d-4c7e-be61-cef08bad5f03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.105225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405a377-481c-4eae-a367-5136bdc34618 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.105500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405a377-481c-4eae-a367-5136bdc34618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.105703 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac913c02-8c2d-4c7e-be61-cef08bad5f03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:15.105810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405a377-481c-4eae-a367-5136bdc34618 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.106384 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2405a377-481c-4eae-a367-5136bdc34618 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:36:15.106680 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2405a377-481c-4eae-a367-5136bdc34618 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:15.106776 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: WARNING py.warnings [None req-ac913c02-8c2d-4c7e-be61-cef08bad5f03 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 10:36:15.106776 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: warnings.warn(msg) Sep 19 10:36:15.106776 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: Sep 19 10:36:15.108419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac913c02-8c2d-4c7e-be61-cef08bad5f03 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:15.108769 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2405a377-481c-4eae-a367-5136bdc34618 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:15.120899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 789/1564] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:36:15.121788 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405a377-481c-4eae-a367-5136bdc34618 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:15.131748 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405a377-481c-4eae-a367-5136bdc34618 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:15.136172 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7497d-2641-4bc5-a5bc-36bd5b934b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:15.136749 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7497d-2641-4bc5-a5bc-36bd5b934b94 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.137023 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7497d-2641-4bc5-a5bc-36bd5b934b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.137321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7497d-2641-4bc5-a5bc-36bd5b934b94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.137947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7497d-2641-4bc5-a5bc-36bd5b934b94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:36:15.138807 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7497d-2641-4bc5-a5bc-36bd5b934b94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:15.140365 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7497d-2641-4bc5-a5bc-36bd5b934b94 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:15.151771 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 790/1565] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:36:15.152644 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7497d-2641-4bc5-a5bc-36bd5b934b94 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:15.162642 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7497d-2641-4bc5-a5bc-36bd5b934b94 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:15.167116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d95548-100e-4204-b875-b6b37074012d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:15.167671 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d95548-100e-4204-b875-b6b37074012d admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.167946 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d95548-100e-4204-b875-b6b37074012d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.168216 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d95548-100e-4204-b875-b6b37074012d admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.168812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d95548-100e-4204-b875-b6b37074012d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:15.170729 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d95548-100e-4204-b875-b6b37074012d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:15.210264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 776/1566] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:15 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:15.210264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac913c02-8c2d-4c7e-be61-cef08bad5f03 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:15.251360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:15.256083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:15.260394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 791/1567] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:15 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:15.260394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d95548-100e-4204-b875-b6b37074012d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:15.271889 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac913c02-8c2d-4c7e-be61-cef08bad5f03 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:15.322253 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d95548-100e-4204-b875-b6b37074012d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:15.327715 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecde252-7626-416f-93df-1cf240d4752c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:15.328007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322cd3a-8249-4808-91ad-c2e9a9dddac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:15.334394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecde252-7626-416f-93df-1cf240d4752c admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.334394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecde252-7626-416f-93df-1cf240d4752c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.334599 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322cd3a-8249-4808-91ad-c2e9a9dddac7 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.334716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecde252-7626-416f-93df-1cf240d4752c admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.334801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322cd3a-8249-4808-91ad-c2e9a9dddac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.334801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322cd3a-8249-4808-91ad-c2e9a9dddac7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.335008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecde252-7626-416f-93df-1cf240d4752c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:15.335110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322cd3a-8249-4808-91ad-c2e9a9dddac7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:36:15.335110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322cd3a-8249-4808-91ad-c2e9a9dddac7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:15.335292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecde252-7626-416f-93df-1cf240d4752c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:15.335395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322cd3a-8249-4808-91ad-c2e9a9dddac7 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:15.342950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 777/1568] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:36:15.343807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322cd3a-8249-4808-91ad-c2e9a9dddac7 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:15.357836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322cd3a-8249-4808-91ad-c2e9a9dddac7 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:15.357836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd490e3-5ad2-435c-b3d8-71c9004adf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:15.358364 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd490e3-5ad2-435c-b3d8-71c9004adf39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.358639 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd490e3-5ad2-435c-b3d8-71c9004adf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.358907 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd490e3-5ad2-435c-b3d8-71c9004adf39 admin admin] PATH_INFO: `/v3/projects/e86736f46b254bc399298305a8888467/users/8a059ba74d744a12a4694513f8ca5789/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.373371 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd490e3-5ad2-435c-b3d8-71c9004adf39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e86736f46b254bc399298305a8888467, user_id=8a059ba74d744a12a4694513f8ca5789, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:15.374386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: WARNING py.warnings [None req-fcd490e3-5ad2-435c-b3d8-71c9004adf39 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 10:36:15.374386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: warnings.warn(msg) Sep 19 10:36:15.374386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: Sep 19 10:36:15.376380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd490e3-5ad2-435c-b3d8-71c9004adf39 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:15.441368 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 792/1569] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:15 2022] POST /identity/v3/projects => generated 403 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:15.442333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecde252-7626-416f-93df-1cf240d4752c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:15.470496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 778/1570] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:15 2022] PUT /identity/v3/projects/e86736f46b254bc399298305a8888467/users/8a059ba74d744a12a4694513f8ca5789/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:15.471421 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd490e3-5ad2-435c-b3d8-71c9004adf39 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:15.503576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecde252-7626-416f-93df-1cf240d4752c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:15.558658 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e05539-0ba6-45cd-9860-c4688ec77dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:15.559271 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e05539-0ba6-45cd-9860-c4688ec77dad admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.559555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e05539-0ba6-45cd-9860-c4688ec77dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.559843 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e05539-0ba6-45cd-9860-c4688ec77dad admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.560976 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e05539-0ba6-45cd-9860-c4688ec77dad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:36:15.561305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e05539-0ba6-45cd-9860-c4688ec77dad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:15.562885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e05539-0ba6-45cd-9860-c4688ec77dad admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:15.569034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd490e3-5ad2-435c-b3d8-71c9004adf39 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:15.573923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac555a9-4c13-4343-a646-fac690b0be9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:15.574108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 793/1571] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:36:15.574538 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac555a9-4c13-4343-a646-fac690b0be9d admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.574778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e05539-0ba6-45cd-9860-c4688ec77dad admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:15.574888 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac555a9-4c13-4343-a646-fac690b0be9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.575138 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac555a9-4c13-4343-a646-fac690b0be9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.576277 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac555a9-4c13-4343-a646-fac690b0be9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:15.578306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac555a9-4c13-4343-a646-fac690b0be9d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:15.593188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e05539-0ba6-45cd-9860-c4688ec77dad admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:15.597827 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99521a7c-81dc-4b37-8438-71c77e9ac946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:15.598429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99521a7c-81dc-4b37-8438-71c77e9ac946 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.598710 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99521a7c-81dc-4b37-8438-71c77e9ac946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.598986 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99521a7c-81dc-4b37-8438-71c77e9ac946 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.599616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99521a7c-81dc-4b37-8438-71c77e9ac946 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:15.601548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99521a7c-81dc-4b37-8438-71c77e9ac946 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:15.669804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 779/1572] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:15 2022] POST /identity/v3/projects => generated 361 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:15.673183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac555a9-4c13-4343-a646-fac690b0be9d admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:15.698784 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 794/1573] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:15 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:15.701711 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99521a7c-81dc-4b37-8438-71c77e9ac946 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:15.704976 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:15.721805 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:15.735386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac555a9-4c13-4343-a646-fac690b0be9d admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:15.763430 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99521a7c-81dc-4b37-8438-71c77e9ac946 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:15.768811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568c350-ed9c-4f1b-859e-bf0faa0b10f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:15.769532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568c350-ed9c-4f1b-859e-bf0faa0b10f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.769804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568c350-ed9c-4f1b-859e-bf0faa0b10f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.770073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568c350-ed9c-4f1b-859e-bf0faa0b10f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.770717 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568c350-ed9c-4f1b-859e-bf0faa0b10f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:15.772647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568c350-ed9c-4f1b-859e-bf0faa0b10f6 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:15.791267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086fa01-73e6-49d7-8f02-c2bb86ef30c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:15.791919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086fa01-73e6-49d7-8f02-c2bb86ef30c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.792251 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086fa01-73e6-49d7-8f02-c2bb86ef30c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.792541 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086fa01-73e6-49d7-8f02-c2bb86ef30c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.793194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086fa01-73e6-49d7-8f02-c2bb86ef30c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:15.795697 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086fa01-73e6-49d7-8f02-c2bb86ef30c9 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:15.836977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 795/1574] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:15 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:15.836977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568c350-ed9c-4f1b-859e-bf0faa0b10f6 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:15.844643 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568c350-ed9c-4f1b-859e-bf0faa0b10f6 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:15.850735 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a9e65-d785-4ca8-b6a5-45d46ac6d657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:15.850735 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a9e65-d785-4ca8-b6a5-45d46ac6d657 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.850735 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a9e65-d785-4ca8-b6a5-45d46ac6d657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.850735 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a9e65-d785-4ca8-b6a5-45d46ac6d657 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.851135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a9e65-d785-4ca8-b6a5-45d46ac6d657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:15.852769 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a9e65-d785-4ca8-b6a5-45d46ac6d657 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:15.903197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 780/1575] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:15 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:15.904295 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086fa01-73e6-49d7-8f02-c2bb86ef30c9 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:15.913887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086fa01-73e6-49d7-8f02-c2bb86ef30c9 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:15.918495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2964d2-99bd-4dcf-9a7f-82d3e629f299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:15.919956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2964d2-99bd-4dcf-9a7f-82d3e629f299 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.919956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2964d2-99bd-4dcf-9a7f-82d3e629f299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.920174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2964d2-99bd-4dcf-9a7f-82d3e629f299 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.920774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2964d2-99bd-4dcf-9a7f-82d3e629f299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:15.922752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2964d2-99bd-4dcf-9a7f-82d3e629f299 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:15.944673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 796/1576] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:15 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:15.945982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52d440-5db8-4bc5-907e-9de6adedf3d9 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:15.946266 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52d440-5db8-4bc5-907e-9de6adedf3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:15.946545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52d440-5db8-4bc5-907e-9de6adedf3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:15.964678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6e52d440-5db8-4bc5-907e-9de6adedf3d9 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:15.965913 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6e52d440-5db8-4bc5-907e-9de6adedf3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:16.004107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 781/1577] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:15 2022] POST /identity/v3/users => generated 302 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:16.005171 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2964d2-99bd-4dcf-9a7f-82d3e629f299 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.015223 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2964d2-99bd-4dcf-9a7f-82d3e629f299 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.025720 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 797/1578] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:16.026665 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52d440-5db8-4bc5-907e-9de6adedf3d9 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.036531 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52d440-5db8-4bc5-907e-9de6adedf3d9 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.041135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4042e-cc4f-407f-9098-afc64faef29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.041741 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4042e-cc4f-407f-9098-afc64faef29a admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.042009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4042e-cc4f-407f-9098-afc64faef29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.042278 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4042e-cc4f-407f-9098-afc64faef29a admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.042899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4042e-cc4f-407f-9098-afc64faef29a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.045623 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4042e-cc4f-407f-9098-afc64faef29a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.080596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeceeaea-b0b3-49a9-b9e5-542bec132965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.081248 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeceeaea-b0b3-49a9-b9e5-542bec132965 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.081537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeceeaea-b0b3-49a9-b9e5-542bec132965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.081801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeceeaea-b0b3-49a9-b9e5-542bec132965 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.082432 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeceeaea-b0b3-49a9-b9e5-542bec132965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.084346 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeceeaea-b0b3-49a9-b9e5-542bec132965 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.139824 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 798/1579] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:16 2022] POST /identity/v3/users => generated 281 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:16.140848 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4042e-cc4f-407f-9098-afc64faef29a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.150456 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4042e-cc4f-407f-9098-afc64faef29a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.155511 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64581fc6-1ff4-4c51-b130-6d8276b67ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.156087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64581fc6-1ff4-4c51-b130-6d8276b67ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.156373 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64581fc6-1ff4-4c51-b130-6d8276b67ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.156650 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64581fc6-1ff4-4c51-b130-6d8276b67ec9 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.157339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64581fc6-1ff4-4c51-b130-6d8276b67ec9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.159274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64581fc6-1ff4-4c51-b130-6d8276b67ec9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.163743 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 782/1580] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:16 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:16.164573 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeceeaea-b0b3-49a9-b9e5-542bec132965 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.215069 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:16.225867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeceeaea-b0b3-49a9-b9e5-542bec132965 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.230550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b457d3-5302-468f-af15-d1a47080e720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.231244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b457d3-5302-468f-af15-d1a47080e720 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.231547 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b457d3-5302-468f-af15-d1a47080e720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.231682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 799/1581] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:16 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:16.231907 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b457d3-5302-468f-af15-d1a47080e720 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.232990 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b457d3-5302-468f-af15-d1a47080e720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.233166 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64581fc6-1ff4-4c51-b130-6d8276b67ec9 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.234999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b457d3-5302-468f-af15-d1a47080e720 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.244341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64581fc6-1ff4-4c51-b130-6d8276b67ec9 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.244791 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 783/1582] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:16.245739 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b457d3-5302-468f-af15-d1a47080e720 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.249611 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a3f98-70f8-4634-9caf-1631455ce324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.250217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a3f98-70f8-4634-9caf-1631455ce324 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.250501 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a3f98-70f8-4634-9caf-1631455ce324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.250773 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a3f98-70f8-4634-9caf-1631455ce324 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.251416 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a3f98-70f8-4634-9caf-1631455ce324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.252994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a3f98-70f8-4634-9caf-1631455ce324 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.256088 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b457d3-5302-468f-af15-d1a47080e720 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.260701 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b2528-8949-468a-b971-fe509d2005be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.261346 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b2528-8949-468a-b971-fe509d2005be admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.263953 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b2528-8949-468a-b971-fe509d2005be admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.263953 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b2528-8949-468a-b971-fe509d2005be admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.263953 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b2528-8949-468a-b971-fe509d2005be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.264240 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 800/1583] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:16.264709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a3f98-70f8-4634-9caf-1631455ce324 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.264835 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b2528-8949-468a-b971-fe509d2005be admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.276940 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a3f98-70f8-4634-9caf-1631455ce324 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.282180 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdfc4e-fa92-4258-b799-4c0c9b46a7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.282768 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdfc4e-fa92-4258-b799-4c0c9b46a7ac admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.283049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdfc4e-fa92-4258-b799-4c0c9b46a7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.283387 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdfc4e-fa92-4258-b799-4c0c9b46a7ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.284052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfdfc4e-fa92-4258-b799-4c0c9b46a7ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.285596 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfdfc4e-fa92-4258-b799-4c0c9b46a7ac admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.296104 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 801/1584] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:16.297052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdfc4e-fa92-4258-b799-4c0c9b46a7ac admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.309667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:16.351791 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 784/1585] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:16 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:16.351791 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b2528-8949-468a-b971-fe509d2005be admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.352060 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdfc4e-fa92-4258-b799-4c0c9b46a7ac admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.355999 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4937e13-fc00-4416-b603-c1b8dfa906d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.356691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4937e13-fc00-4416-b603-c1b8dfa906d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.356919 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4937e13-fc00-4416-b603-c1b8dfa906d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.357742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4937e13-fc00-4416-b603-c1b8dfa906d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.362167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b2528-8949-468a-b971-fe509d2005be admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.368168 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4937e13-fc00-4416-b603-c1b8dfa906d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.368602 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6137e2f-7881-447a-97c8-e6b22b51cd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.369228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6137e2f-7881-447a-97c8-e6b22b51cd77 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.369528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6137e2f-7881-447a-97c8-e6b22b51cd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.369642 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4937e13-fc00-4416-b603-c1b8dfa906d4 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.369799 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6137e2f-7881-447a-97c8-e6b22b51cd77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.370433 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6137e2f-7881-447a-97c8-e6b22b51cd77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.372049 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6137e2f-7881-447a-97c8-e6b22b51cd77 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.379148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 802/1586] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:16.379932 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4937e13-fc00-4416-b603-c1b8dfa906d4 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.386485 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 785/1587] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:16.386485 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6137e2f-7881-447a-97c8-e6b22b51cd77 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.392893 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4937e13-fc00-4416-b603-c1b8dfa906d4 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.395707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6137e2f-7881-447a-97c8-e6b22b51cd77 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.398118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff89a2-7b50-451b-9087-ff9de04162bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.398782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff89a2-7b50-451b-9087-ff9de04162bd admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.399068 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff89a2-7b50-451b-9087-ff9de04162bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.399344 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff89a2-7b50-451b-9087-ff9de04162bd admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.399978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff89a2-7b50-451b-9087-ff9de04162bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.401411 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4bcef-b171-489a-a9a2-eb928034f331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.402055 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff89a2-7b50-451b-9087-ff9de04162bd admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.402173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4bcef-b171-489a-a9a2-eb928034f331 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.402338 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4bcef-b171-489a-a9a2-eb928034f331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.402609 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4bcef-b171-489a-a9a2-eb928034f331 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.403276 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4bcef-b171-489a-a9a2-eb928034f331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.405674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4bcef-b171-489a-a9a2-eb928034f331 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.416074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 786/1588] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:16.417028 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4bcef-b171-489a-a9a2-eb928034f331 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.426471 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4bcef-b171-489a-a9a2-eb928034f331 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.431119 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe4573-1bcc-4acb-a940-e386ebb1a4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.431682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe4573-1bcc-4acb-a940-e386ebb1a4c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.431998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe4573-1bcc-4acb-a940-e386ebb1a4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.432276 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe4573-1bcc-4acb-a940-e386ebb1a4c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.432890 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe4573-1bcc-4acb-a940-e386ebb1a4c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.434471 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe4573-1bcc-4acb-a940-e386ebb1a4c9 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.444120 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 787/1589] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:16.444867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe4573-1bcc-4acb-a940-e386ebb1a4c9 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.454409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe4573-1bcc-4acb-a940-e386ebb1a4c9 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.458910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9621e59-1060-4de1-abc6-7f6eb711f749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.459444 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9621e59-1060-4de1-abc6-7f6eb711f749 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.459725 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9621e59-1060-4de1-abc6-7f6eb711f749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.460017 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9621e59-1060-4de1-abc6-7f6eb711f749 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.460671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9621e59-1060-4de1-abc6-7f6eb711f749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.462258 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9621e59-1060-4de1-abc6-7f6eb711f749 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.473528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 788/1590] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:16.474050 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9621e59-1060-4de1-abc6-7f6eb711f749 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.483560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9621e59-1060-4de1-abc6-7f6eb711f749 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.488128 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c2e81-07d5-4aa6-a78e-5b64d1444330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.488801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c2e81-07d5-4aa6-a78e-5b64d1444330 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.489117 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c2e81-07d5-4aa6-a78e-5b64d1444330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.489397 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c2e81-07d5-4aa6-a78e-5b64d1444330 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.490013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c2e81-07d5-4aa6-a78e-5b64d1444330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.491587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c2e81-07d5-4aa6-a78e-5b64d1444330 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.501756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 789/1591] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:16.502495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c2e81-07d5-4aa6-a78e-5b64d1444330 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.512030 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c2e81-07d5-4aa6-a78e-5b64d1444330 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.519826 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64de5f-f5ee-4858-a418-42a4ae35cdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.520691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64de5f-f5ee-4858-a418-42a4ae35cdbc admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.521157 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64de5f-f5ee-4858-a418-42a4ae35cdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.521611 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64de5f-f5ee-4858-a418-42a4ae35cdbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.522407 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64de5f-f5ee-4858-a418-42a4ae35cdbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.524254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64de5f-f5ee-4858-a418-42a4ae35cdbc admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.527415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 803/1592] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:16 2022] POST /identity/v3/users => generated 288 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:16.536607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff89a2-7b50-451b-9087-ff9de04162bd admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.538003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 790/1593] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:16.538931 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64de5f-f5ee-4858-a418-42a4ae35cdbc admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.542705 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff89a2-7b50-451b-9087-ff9de04162bd admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.547479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af25aa0-8116-47e0-8d59-012dcbbf6bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.548101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af25aa0-8116-47e0-8d59-012dcbbf6bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.548385 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af25aa0-8116-47e0-8d59-012dcbbf6bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.548659 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af25aa0-8116-47e0-8d59-012dcbbf6bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.549220 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64de5f-f5ee-4858-a418-42a4ae35cdbc admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.549359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af25aa0-8116-47e0-8d59-012dcbbf6bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.550936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af25aa0-8116-47e0-8d59-012dcbbf6bd7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.554542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ebe92f-f0cf-4eb9-ad26-d5561789401d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.555227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebe92f-f0cf-4eb9-ad26-d5561789401d admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.555603 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebe92f-f0cf-4eb9-ad26-d5561789401d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.555770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebe92f-f0cf-4eb9-ad26-d5561789401d admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.557151 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ebe92f-f0cf-4eb9-ad26-d5561789401d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.558241 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ebe92f-f0cf-4eb9-ad26-d5561789401d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.562681 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 804/1594] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:16.564649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af25aa0-8116-47e0-8d59-012dcbbf6bd7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.570591 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 791/1595] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:16.571062 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ebe92f-f0cf-4eb9-ad26-d5561789401d admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.580098 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af25aa0-8116-47e0-8d59-012dcbbf6bd7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.581629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ebe92f-f0cf-4eb9-ad26-d5561789401d admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.587610 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbd3c5-a099-4b6b-873a-76317f0d18d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.587610 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbd3c5-a099-4b6b-873a-76317f0d18d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.587610 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbd3c5-a099-4b6b-873a-76317f0d18d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.588068 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6927a7-3665-4350-abf7-3cb27cf4e07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.588200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbd3c5-a099-4b6b-873a-76317f0d18d3 admin admin] PATH_INFO: `/v3/projects/e3160b641b81433780d38eca696bfdf1/users/d4d9bea213fa4401992d8e7894726e05/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.588289 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6927a7-3665-4350-abf7-3cb27cf4e07d admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.588289 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6927a7-3665-4350-abf7-3cb27cf4e07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.588289 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6927a7-3665-4350-abf7-3cb27cf4e07d admin admin] PATH_INFO: `/v3/projects/67ec80c3417443d7b2368506bbd4a814/users/6712334ebb77462daa89be58d4480846/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.603701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbd3c5-a099-4b6b-873a-76317f0d18d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3160b641b81433780d38eca696bfdf1, user_id=d4d9bea213fa4401992d8e7894726e05, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.605623 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6927a7-3665-4350-abf7-3cb27cf4e07d admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ec80c3417443d7b2368506bbd4a814, user_id=6712334ebb77462daa89be58d4480846, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.606529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbd3c5-a099-4b6b-873a-76317f0d18d3 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.610553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6927a7-3665-4350-abf7-3cb27cf4e07d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.663108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 805/1596] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:16 2022] PUT /identity/v3/projects/e3160b641b81433780d38eca696bfdf1/users/d4d9bea213fa4401992d8e7894726e05/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:16.667296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 792/1597] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:16 2022] PUT /identity/v3/projects/67ec80c3417443d7b2368506bbd4a814/users/6712334ebb77462daa89be58d4480846/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:16.667296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6927a7-3665-4350-abf7-3cb27cf4e07d admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.668027 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbd3c5-a099-4b6b-873a-76317f0d18d3 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.745907 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6927a7-3665-4350-abf7-3cb27cf4e07d admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.746779 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbd3c5-a099-4b6b-873a-76317f0d18d3 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.751012 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9880325-3072-412d-a9a7-a1dba2d16b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.751624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9880325-3072-412d-a9a7-a1dba2d16b2f admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.751973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9880325-3072-412d-a9a7-a1dba2d16b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.752161 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9880325-3072-412d-a9a7-a1dba2d16b2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.752685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5179e30-201e-4d92-a936-e6fb8ba7c945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.752821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9880325-3072-412d-a9a7-a1dba2d16b2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.754115 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5179e30-201e-4d92-a936-e6fb8ba7c945 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.754115 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5179e30-201e-4d92-a936-e6fb8ba7c945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.754417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5179e30-201e-4d92-a936-e6fb8ba7c945 admin admin] PATH_INFO: `/v3/projects/9ef6a248373c47c0b7af49a1833c017a/users/2895914d8bcc4960a646eac3df8d92f3/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.754539 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9880325-3072-412d-a9a7-a1dba2d16b2f admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.767273 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 793/1598] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:16.768523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9880325-3072-412d-a9a7-a1dba2d16b2f admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.775629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5179e30-201e-4d92-a936-e6fb8ba7c945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ef6a248373c47c0b7af49a1833c017a, user_id=2895914d8bcc4960a646eac3df8d92f3, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.778221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9880325-3072-412d-a9a7-a1dba2d16b2f admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.778394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5179e30-201e-4d92-a936-e6fb8ba7c945 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.783276 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517f1d0-6221-4ccf-a1cf-ed9df218fb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.783891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517f1d0-6221-4ccf-a1cf-ed9df218fb87 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.784207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517f1d0-6221-4ccf-a1cf-ed9df218fb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.784481 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517f1d0-6221-4ccf-a1cf-ed9df218fb87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.785131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517f1d0-6221-4ccf-a1cf-ed9df218fb87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.786692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517f1d0-6221-4ccf-a1cf-ed9df218fb87 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.797158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 794/1599] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:16.798091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517f1d0-6221-4ccf-a1cf-ed9df218fb87 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.807668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517f1d0-6221-4ccf-a1cf-ed9df218fb87 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.814985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2283e18-9849-4f95-8c09-045b179b7971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.815687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2283e18-9849-4f95-8c09-045b179b7971 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.816024 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2283e18-9849-4f95-8c09-045b179b7971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.816302 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2283e18-9849-4f95-8c09-045b179b7971 admin admin] PATH_INFO: `/v3/projects/ff0da561855b48c7908860c72039c112/users/5fac876a606247808751df1ec5efcc9f/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.829124 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:16.832184 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2283e18-9849-4f95-8c09-045b179b7971 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff0da561855b48c7908860c72039c112, user_id=5fac876a606247808751df1ec5efcc9f, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.834905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2283e18-9849-4f95-8c09-045b179b7971 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.874127 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 806/1600] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:16 2022] PUT /identity/v3/projects/9ef6a248373c47c0b7af49a1833c017a/users/2895914d8bcc4960a646eac3df8d92f3/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:16.875407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5179e30-201e-4d92-a936-e6fb8ba7c945 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:16.924402 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:16.939284 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5179e30-201e-4d92-a936-e6fb8ba7c945 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:16.944316 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5187b-13fc-4efa-a98e-1c5dd002a35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:16.951828 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5187b-13fc-4efa-a98e-1c5dd002a35f admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.951828 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5187b-13fc-4efa-a98e-1c5dd002a35f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.951828 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5187b-13fc-4efa-a98e-1c5dd002a35f admin admin] PATH_INFO: `/v3/projects/2998988ae7ae429ca62921ac188de9b9/users/01008443277843d5b61807b64bdd8c72/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.960893 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5187b-13fc-4efa-a98e-1c5dd002a35f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2998988ae7ae429ca62921ac188de9b9, user_id=01008443277843d5b61807b64bdd8c72, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:16.963589 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5187b-13fc-4efa-a98e-1c5dd002a35f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:16.975106 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 795/1601] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:16 2022] PUT /identity/v3/projects/ff0da561855b48c7908860c72039c112/users/5fac876a606247808751df1ec5efcc9f/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:16.976876 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a06e8-95ff-483d-a092-a2055171048b None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:16.977307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a06e8-95ff-483d-a092-a2055171048b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:16.977684 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a06e8-95ff-483d-a092-a2055171048b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:16.997173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-9c3a06e8-95ff-483d-a092-a2055171048b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:16.998563 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-9c3a06e8-95ff-483d-a092-a2055171048b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:17.060631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 796/1602] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:17.061910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9431d9-7e94-4a98-9252-5cf64189de8e None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:17.062195 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9431d9-7e94-4a98-9252-5cf64189de8e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:17.062473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9431d9-7e94-4a98-9252-5cf64189de8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:17.080986 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-8d9431d9-7e94-4a98-9252-5cf64189de8e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:17.082255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-8d9431d9-7e94-4a98-9252-5cf64189de8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:17.090006 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 797/1603] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:17.090927 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9431d9-7e94-4a98-9252-5cf64189de8e None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:17.100939 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9431d9-7e94-4a98-9252-5cf64189de8e None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:17.102961 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 807/1604] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:16 2022] PUT /identity/v3/projects/2998988ae7ae429ca62921ac188de9b9/users/01008443277843d5b61807b64bdd8c72/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:17.103773 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5187b-13fc-4efa-a98e-1c5dd002a35f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:17.164382 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee2dd7-6be3-4efe-810d-123edf83b6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:17.165337 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee2dd7-6be3-4efe-810d-123edf83b6ee admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:17.165799 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee2dd7-6be3-4efe-810d-123edf83b6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:17.166520 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee2dd7-6be3-4efe-810d-123edf83b6ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:17.167329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee2dd7-6be3-4efe-810d-123edf83b6ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:17.169130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee2dd7-6be3-4efe-810d-123edf83b6ee admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:17.180186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 798/1605] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:17.183563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5187b-13fc-4efa-a98e-1c5dd002a35f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:17.185047 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175a012-1aec-417f-a765-100738f74a6d None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:17.185358 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175a012-1aec-417f-a765-100738f74a6d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:17.185632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175a012-1aec-417f-a765-100738f74a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:17.187942 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a184606-1cbb-4188-b538-6ea6aeab8808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:17.188684 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a184606-1cbb-4188-b538-6ea6aeab8808 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:17.189037 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a184606-1cbb-4188-b538-6ea6aeab8808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:17.189454 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a184606-1cbb-4188-b538-6ea6aeab8808 admin admin] PATH_INFO: `/v3/projects/a48df5556fc247fba5eaf7d3d1ecf44f/users/5fecaefd115842fe80009d70921c3806/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:17.205460 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-7175a012-1aec-417f-a765-100738f74a6d None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:17.209186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-7175a012-1aec-417f-a765-100738f74a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:17.209360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a184606-1cbb-4188-b538-6ea6aeab8808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a48df5556fc247fba5eaf7d3d1ecf44f, user_id=5fecaefd115842fe80009d70921c3806, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:17.211057 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a184606-1cbb-4188-b538-6ea6aeab8808 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:17.215073 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 799/1606] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:17.216118 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785ad37-3fa8-473c-bdda-940c6fa11d4d None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:17.217053 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785ad37-3fa8-473c-bdda-940c6fa11d4d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:17.217374 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785ad37-3fa8-473c-bdda-940c6fa11d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:17.246291 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-7785ad37-3fa8-473c-bdda-940c6fa11d4d None None] MFA Rules not processed for user `5fac876a606247808751df1ec5efcc9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:17.247663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-7785ad37-3fa8-473c-bdda-940c6fa11d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:17.294509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 808/1607] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:17 2022] PUT /identity/v3/projects/a48df5556fc247fba5eaf7d3d1ecf44f/users/5fecaefd115842fe80009d70921c3806/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:17.294509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a184606-1cbb-4188-b538-6ea6aeab8808 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:17.312686 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 800/1608] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:17.313582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785ad37-3fa8-473c-bdda-940c6fa11d4d None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:17.318736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:17.352754 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a184606-1cbb-4188-b538-6ea6aeab8808 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:17.357362 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4880e-1d74-4264-a2ba-a8aacbfa5372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:17.357982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4880e-1d74-4264-a2ba-a8aacbfa5372 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:17.358265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4880e-1d74-4264-a2ba-a8aacbfa5372 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:17.358543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4880e-1d74-4264-a2ba-a8aacbfa5372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:17.370913 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785ad37-3fa8-473c-bdda-940c6fa11d4d None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:17.377237 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa51c19-0e51-4901-9b9d-bee5edb355d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:17.382984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51c19-0e51-4901-9b9d-bee5edb355d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:17.382984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51c19-0e51-4901-9b9d-bee5edb355d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:17.382984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51c19-0e51-4901-9b9d-bee5edb355d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:17.439677 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4880e-1d74-4264-a2ba-a8aacbfa5372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:17.446299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4880e-1d74-4264-a2ba-a8aacbfa5372 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:17.446425 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa51c19-0e51-4901-9b9d-bee5edb355d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:17.446425 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa51c19-0e51-4901-9b9d-bee5edb355d2 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:17.452955 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 809/1609] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:17.454357 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7ffcf-0824-48eb-9a12-2084869e6950 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:17.454691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7ffcf-0824-48eb-9a12-2084869e6950 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:17.455159 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7ffcf-0824-48eb-9a12-2084869e6950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:17.455900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 801/1610] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:17.456563 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa51c19-0e51-4901-9b9d-bee5edb355d2 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:17.463301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:17.465768 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa51c19-0e51-4901-9b9d-bee5edb355d2 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:17.474740 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069004e-75ed-4a65-9026-04e1cdb0f486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:17.474740 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069004e-75ed-4a65-9026-04e1cdb0f486 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:17.474740 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069004e-75ed-4a65-9026-04e1cdb0f486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:17.474740 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069004e-75ed-4a65-9026-04e1cdb0f486 admin admin] PATH_INFO: `/v3/projects/dc87da0661ec4a6caef7e46c13a39482/users/7f2807b7c7db45848fe6184c9134fc21/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:17.490810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-56c7ffcf-0824-48eb-9a12-2084869e6950 None None] MFA Rules not processed for user `01008443277843d5b61807b64bdd8c72`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:17.492196 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-56c7ffcf-0824-48eb-9a12-2084869e6950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:17.492788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069004e-75ed-4a65-9026-04e1cdb0f486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc87da0661ec4a6caef7e46c13a39482, user_id=7f2807b7c7db45848fe6184c9134fc21, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:17.495830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069004e-75ed-4a65-9026-04e1cdb0f486 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:17.557687 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 810/1611] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:17.558548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7ffcf-0824-48eb-9a12-2084869e6950 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:17.559821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-56c7ffcf-0824-48eb-9a12-2084869e6950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:17.570975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7ffcf-0824-48eb-9a12-2084869e6950 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:17.574620 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884ec99-b684-4360-995c-22a4b7ea740c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:17.575209 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884ec99-b684-4360-995c-22a4b7ea740c service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:17.575514 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884ec99-b684-4360-995c-22a4b7ea740c service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:17.575787 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884ec99-b684-4360-995c-22a4b7ea740c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:17.585302 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884ec99-b684-4360-995c-22a4b7ea740c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:17.589526 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884ec99-b684-4360-995c-22a4b7ea740c service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:17.597525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 811/1612] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:17.598993 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce0851-c8fe-4ba3-98bd-537a6805813c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:17.599451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce0851-c8fe-4ba3-98bd-537a6805813c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:17.600044 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce0851-c8fe-4ba3-98bd-537a6805813c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:17.620397 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6fce0851-c8fe-4ba3-98bd-537a6805813c None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:17.621650 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6fce0851-c8fe-4ba3-98bd-537a6805813c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:17.627330 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 802/1613] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:17 2022] PUT /identity/v3/projects/dc87da0661ec4a6caef7e46c13a39482/users/7f2807b7c7db45848fe6184c9134fc21/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:17.635902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069004e-75ed-4a65-9026-04e1cdb0f486 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:17.702342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 812/1614] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:17.704366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031addb-6cd3-4a74-ba26-bf080965658a None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:17.704668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031addb-6cd3-4a74-ba26-bf080965658a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:17.704943 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031addb-6cd3-4a74-ba26-bf080965658a None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:17.715281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069004e-75ed-4a65-9026-04e1cdb0f486 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:17.716944 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 813/1615] 10.222.0.42 () {54 vars in 965 bytes} [Mon Sep 19 10:36:17 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:36:17.718354 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccc659-66df-4fb7-b17a-e807c97a8e5e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:17.718630 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccc659-66df-4fb7-b17a-e807c97a8e5e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:17.718909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccc659-66df-4fb7-b17a-e807c97a8e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:17.730736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cdf61-edad-43ad-8fd1-70331ccc85cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:17.731355 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cdf61-edad-43ad-8fd1-70331ccc85cb admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:17.731643 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cdf61-edad-43ad-8fd1-70331ccc85cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:17.732582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cdf61-edad-43ad-8fd1-70331ccc85cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:17.733268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56cdf61-edad-43ad-8fd1-70331ccc85cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:17.736078 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56cdf61-edad-43ad-8fd1-70331ccc85cb admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:17.777559 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a9ccc659-66df-4fb7-b17a-e807c97a8e5e None None] MFA Rules not processed for user `7f2807b7c7db45848fe6184c9134fc21`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:17.779545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a9ccc659-66df-4fb7-b17a-e807c97a8e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:17.822608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 803/1616] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:17 2022] POST /identity/v3/projects => generated 363 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:17.824476 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d6d69-a91e-42c8-9077-96696cfcbe84 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:17.825278 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d6d69-a91e-42c8-9077-96696cfcbe84 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:17.825278 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d6d69-a91e-42c8-9077-96696cfcbe84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:17.886920 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:17.897942 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 814/1617] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:17.900106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccc659-66df-4fb7-b17a-e807c97a8e5e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:17.925321 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-9b1d6d69-a91e-42c8-9077-96696cfcbe84 None None] MFA Rules not processed for user `b4007794a30247c4a318af62d05e3b92`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:17.926555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:17.930237 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-9b1d6d69-a91e-42c8-9077-96696cfcbe84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:18.023952 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccc659-66df-4fb7-b17a-e807c97a8e5e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:18.049969 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 804/1618] 10.222.0.42 () {58 vars in 1062 bytes} [Mon Sep 19 10:36:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:18.051126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d6d69-a91e-42c8-9077-96696cfcbe84 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:18.116184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0962da-4bbe-4700-8917-e8d91d43839b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:18.116830 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0962da-4bbe-4700-8917-e8d91d43839b service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:18.117130 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0962da-4bbe-4700-8917-e8d91d43839b service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:18.117414 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0962da-4bbe-4700-8917-e8d91d43839b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:18.161938 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d6d69-a91e-42c8-9077-96696cfcbe84 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:18.172318 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02ffa3-88dd-4b0f-a67d-47efc316ce71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:18.173198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02ffa3-88dd-4b0f-a67d-47efc316ce71 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:18.173699 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02ffa3-88dd-4b0f-a67d-47efc316ce71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:18.174070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02ffa3-88dd-4b0f-a67d-47efc316ce71 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:18.175290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02ffa3-88dd-4b0f-a67d-47efc316ce71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:18.179181 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02ffa3-88dd-4b0f-a67d-47efc316ce71 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:18.225773 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0962da-4bbe-4700-8917-e8d91d43839b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:18.227195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0962da-4bbe-4700-8917-e8d91d43839b service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:18.236938 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 815/1619] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:18.238076 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24555529-8a40-48f1-b7bb-7584e566fb6f None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:18.238364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24555529-8a40-48f1-b7bb-7584e566fb6f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:18.238641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24555529-8a40-48f1-b7bb-7584e566fb6f None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:18.239989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 816/1620] 10.222.0.42 () {54 vars in 965 bytes} [Mon Sep 19 10:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:36:18.242042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24555529-8a40-48f1-b7bb-7584e566fb6f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:18.264764 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24555529-8a40-48f1-b7bb-7584e566fb6f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:18.272174 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d2916-74d3-44b6-96c6-1fc1873135e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:18.272794 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d2916-74d3-44b6-96c6-1fc1873135e1 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:18.273104 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d2916-74d3-44b6-96c6-1fc1873135e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:18.273388 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d2916-74d3-44b6-96c6-1fc1873135e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:18.304301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 805/1621] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:18 2022] POST /identity/v3/users => generated 288 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:18.306598 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aff400-02ed-4c45-96c4-565ff1200db4 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:18.307394 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aff400-02ed-4c45-96c4-565ff1200db4 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:18.307616 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aff400-02ed-4c45-96c4-565ff1200db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:18.333671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-81aff400-02ed-4c45-96c4-565ff1200db4 None None] MFA Rules not processed for user `b4007794a30247c4a318af62d05e3b92`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:18.335069 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-81aff400-02ed-4c45-96c4-565ff1200db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:18.339266 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d2916-74d3-44b6-96c6-1fc1873135e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:18.341615 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d2916-74d3-44b6-96c6-1fc1873135e1 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:18.356835 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 806/1622] 10.222.0.42 () {58 vars in 1062 bytes} [Mon Sep 19 10:36:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:18.358998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aff400-02ed-4c45-96c4-565ff1200db4 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:18.359135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 817/1623] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:18.361279 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d2916-74d3-44b6-96c6-1fc1873135e1 service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:18.371063 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aff400-02ed-4c45-96c4-565ff1200db4 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:18.376646 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d2916-74d3-44b6-96c6-1fc1873135e1 service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:18.376800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20beec-8f09-4d67-a607-5f6a403d7aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:18.377520 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20beec-8f09-4d67-a607-5f6a403d7aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:18.378036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20beec-8f09-4d67-a607-5f6a403d7aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:18.378386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20beec-8f09-4d67-a607-5f6a403d7aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:18.379400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f20beec-8f09-4d67-a607-5f6a403d7aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:18.381143 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f20beec-8f09-4d67-a607-5f6a403d7aa5 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:18.384953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6099df-d049-4c57-9d05-9e06dc823a28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:18.385751 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6099df-d049-4c57-9d05-9e06dc823a28 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:18.386248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6099df-d049-4c57-9d05-9e06dc823a28 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:18.386895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6099df-d049-4c57-9d05-9e06dc823a28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:18.396214 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 807/1624] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:18.400652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:18.401921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20beec-8f09-4d67-a607-5f6a403d7aa5 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:18.417457 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20beec-8f09-4d67-a607-5f6a403d7aa5 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:18.427966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad85a05-0729-4d27-8074-089933e898a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:18.429036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad85a05-0729-4d27-8074-089933e898a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:18.430791 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad85a05-0729-4d27-8074-089933e898a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:18.431963 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad85a05-0729-4d27-8074-089933e898a0 admin admin] PATH_INFO: `/v3/projects/38b637de610d401483dbfa2f42db92b3/users/1e44d6f9bbd340a89f8574039194f3c2/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:18.472426 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad85a05-0729-4d27-8074-089933e898a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b637de610d401483dbfa2f42db92b3, user_id=1e44d6f9bbd340a89f8574039194f3c2, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:18.475754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad85a05-0729-4d27-8074-089933e898a0 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:18.505605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6099df-d049-4c57-9d05-9e06dc823a28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:18.512525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6099df-d049-4c57-9d05-9e06dc823a28 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:18.531335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 818/1625] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:18.616422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 808/1626] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:18 2022] PUT /identity/v3/projects/38b637de610d401483dbfa2f42db92b3/users/1e44d6f9bbd340a89f8574039194f3c2/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:18.616829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6099df-d049-4c57-9d05-9e06dc823a28 service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:18.631657 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6099df-d049-4c57-9d05-9e06dc823a28 service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:18.715135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854722c-ba0a-4dcc-a217-21548ffc8e3d None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:18.715465 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854722c-ba0a-4dcc-a217-21548ffc8e3d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:18.715705 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854722c-ba0a-4dcc-a217-21548ffc8e3d None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:18.717318 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 809/1627] 10.222.0.42 () {54 vars in 912 bytes} [Mon Sep 19 10:36:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:36:18.723497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b5a45-84db-47dc-ab90-80f9483d4a84 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:18.723497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b5a45-84db-47dc-ab90-80f9483d4a84 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:18.723776 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b5a45-84db-47dc-ab90-80f9483d4a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:18.760688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-fa1b5a45-84db-47dc-ab90-80f9483d4a84 None None] MFA Rules not processed for user `b4007794a30247c4a318af62d05e3b92`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:18.762120 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-fa1b5a45-84db-47dc-ab90-80f9483d4a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:18.777854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2452d58-ec3f-4968-acf0-731ac8e8572f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:18.778677 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2452d58-ec3f-4968-acf0-731ac8e8572f admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:18.778970 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2452d58-ec3f-4968-acf0-731ac8e8572f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:18.779244 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2452d58-ec3f-4968-acf0-731ac8e8572f admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:18.780271 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2452d58-ec3f-4968-acf0-731ac8e8572f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:18.781609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2452d58-ec3f-4968-acf0-731ac8e8572f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:18.809594 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 819/1628] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:18.812442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2452d58-ec3f-4968-acf0-731ac8e8572f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:18.836512 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2452d58-ec3f-4968-acf0-731ac8e8572f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:18.858439 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a560e2a-8eaa-42f3-b0ef-e8532aa52402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:18.859514 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a560e2a-8eaa-42f3-b0ef-e8532aa52402 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:18.860004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a560e2a-8eaa-42f3-b0ef-e8532aa52402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:18.860296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a560e2a-8eaa-42f3-b0ef-e8532aa52402 admin admin] PATH_INFO: `/v3/projects/38b637de610d401483dbfa2f42db92b3/users/1e44d6f9bbd340a89f8574039194f3c2/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:18.863215 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a560e2a-8eaa-42f3-b0ef-e8532aa52402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b637de610d401483dbfa2f42db92b3, user_id=1e44d6f9bbd340a89f8574039194f3c2, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:18.866570 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a560e2a-8eaa-42f3-b0ef-e8532aa52402 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:18.888953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 820/1629] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:18 2022] PUT /identity/v3/projects/38b637de610d401483dbfa2f42db92b3/users/1e44d6f9bbd340a89f8574039194f3c2/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:36:18.901007 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fdf64-c3ff-4eda-9276-786422143201 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:18.901457 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fdf64-c3ff-4eda-9276-786422143201 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:18.901802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fdf64-c3ff-4eda-9276-786422143201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:18.950808 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 810/1630] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 19 10:36:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:36:18.955188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:18.961664 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b5a45-84db-47dc-ab90-80f9483d4a84 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:18.979777 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-488fdf64-c3ff-4eda-9276-786422143201 None None] MFA Rules not processed for user `1e44d6f9bbd340a89f8574039194f3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:18.988073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-488fdf64-c3ff-4eda-9276-786422143201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:19.077261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b5a45-84db-47dc-ab90-80f9483d4a84 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:19.122036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 821/1631] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:36:19.124416 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98183ca-e3d5-481b-a605-f006bc3475d5 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:19.124771 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98183ca-e3d5-481b-a605-f006bc3475d5 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:19.125192 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98183ca-e3d5-481b-a605-f006bc3475d5 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:19.132022 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 822/1632] 10.222.0.42 () {54 vars in 912 bytes} [Mon Sep 19 10:36:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:36:19.133038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98183ca-e3d5-481b-a605-f006bc3475d5 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:19.168704 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d318cce-be8e-4f7b-b693-99ccb69e4c7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:19.170843 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:19.171632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d318cce-be8e-4f7b-b693-99ccb69e4c7e service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:19.171866 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d318cce-be8e-4f7b-b693-99ccb69e4c7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:19.172809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d318cce-be8e-4f7b-b693-99ccb69e4c7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:19.680970 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98183ca-e3d5-481b-a605-f006bc3475d5 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:19.680970 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7c365-9aad-4312-b983-3d8777f3b4a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:19.680970 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7c365-9aad-4312-b983-3d8777f3b4a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:19.680970 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7c365-9aad-4312-b983-3d8777f3b4a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:19.691075 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7c365-9aad-4312-b983-3d8777f3b4a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:19.691075 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b7c365-9aad-4312-b983-3d8777f3b4a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:19.691075 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b7c365-9aad-4312-b983-3d8777f3b4a8 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:19.692368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d318cce-be8e-4f7b-b693-99ccb69e4c7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:19.692368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d318cce-be8e-4f7b-b693-99ccb69e4c7e service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:19.693933 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 823/1633] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:19.694348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 811/1634] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:36:19.701187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d9234-b6c2-432b-a251-d923f82421e4 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:19.701187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d9234-b6c2-432b-a251-d923f82421e4 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:19.701187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d9234-b6c2-432b-a251-d923f82421e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:19.701187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f55d9234-b6c2-432b-a251-d923f82421e4 None None] MFA Rules not processed for user `b4007794a30247c4a318af62d05e3b92`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:19.701187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f55d9234-b6c2-432b-a251-d923f82421e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:19.701187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 824/1635] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 19 10:36:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:19.731632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d318cce-be8e-4f7b-b693-99ccb69e4c7e service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:19.743306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d318cce-be8e-4f7b-b693-99ccb69e4c7e service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:19.754952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d464f-67c5-44ba-9578-d821b085488a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:19.755716 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d464f-67c5-44ba-9578-d821b085488a service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:19.755918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d464f-67c5-44ba-9578-d821b085488a service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:19.756189 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d464f-67c5-44ba-9578-d821b085488a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:19.767872 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d464f-67c5-44ba-9578-d821b085488a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:19.769767 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d464f-67c5-44ba-9578-d821b085488a service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:19.782725 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 812/1636] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:20.151968 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00df05-abea-4868-aee5-1bc2eb8a2e88 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:20.152482 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00df05-abea-4868-aee5-1bc2eb8a2e88 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:20.152902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00df05-abea-4868-aee5-1bc2eb8a2e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:20.202526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d464f-67c5-44ba-9578-d821b085488a service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:20.257457 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-be00df05-abea-4868-aee5-1bc2eb8a2e88 None None] MFA Rules not processed for user `2895914d8bcc4960a646eac3df8d92f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:20.258761 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-be00df05-abea-4868-aee5-1bc2eb8a2e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:20.358240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:20.363942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d464f-67c5-44ba-9578-d821b085488a service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:20.369214 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b36e8-c580-4949-8adc-cb0887e0b8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:20.370095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b36e8-c580-4949-8adc-cb0887e0b8c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:20.370823 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b36e8-c580-4949-8adc-cb0887e0b8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:20.371243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b36e8-c580-4949-8adc-cb0887e0b8c9 admin admin] PATH_INFO: `/v3/users/7f2807b7c7db45848fe6184c9134fc21` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:20.374011 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b36e8-c580-4949-8adc-cb0887e0b8c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f2807b7c7db45848fe6184c9134fc21)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:20.375992 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: WARNING py.warnings [None req-794b36e8-c580-4949-8adc-cb0887e0b8c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 10:36:20.375992 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: warnings.warn(msg) Sep 19 10:36:20.375992 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: Sep 19 10:36:20.377632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b36e8-c580-4949-8adc-cb0887e0b8c9 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:20.391008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 825/1637] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:20 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:20.396278 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00df05-abea-4868-aee5-1bc2eb8a2e88 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:20.429671 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00df05-abea-4868-aee5-1bc2eb8a2e88 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:20.451430 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b251e0-c5d0-4483-b7b4-84a986e534b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:20.452078 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b251e0-c5d0-4483-b7b4-84a986e534b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:20.452364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b251e0-c5d0-4483-b7b4-84a986e534b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:20.452640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b251e0-c5d0-4483-b7b4-84a986e534b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:20.453321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b251e0-c5d0-4483-b7b4-84a986e534b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:20.456718 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b251e0-c5d0-4483-b7b4-84a986e534b9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:20.618769 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 826/1638] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:20 2022] POST /identity/v3/projects => generated 405 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:20.627777 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b251e0-c5d0-4483-b7b4-84a986e534b9 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:20.759543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b251e0-c5d0-4483-b7b4-84a986e534b9 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:20.779824 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d09445-5bd4-482c-b2d3-22239465395b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:20.782568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d09445-5bd4-482c-b2d3-22239465395b admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:20.782568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d09445-5bd4-482c-b2d3-22239465395b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:20.782568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d09445-5bd4-482c-b2d3-22239465395b admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:20.783611 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d09445-5bd4-482c-b2d3-22239465395b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:20.786049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d09445-5bd4-482c-b2d3-22239465395b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:20.799169 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-794b36e8-c580-4949-8adc-cb0887e0b8c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f2807b7c7db45848fe6184c9134fc21'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:20.874465 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-794b36e8-c580-4949-8adc-cb0887e0b8c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f2807b7c7db45848fe6184c9134fc21'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:20.875104 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-794b36e8-c580-4949-8adc-cb0887e0b8c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f2807b7c7db45848fe6184c9134fc21'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:20.875725 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-794b36e8-c580-4949-8adc-cb0887e0b8c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f2807b7c7db45848fe6184c9134fc21'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:20.893872 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-794b36e8-c580-4949-8adc-cb0887e0b8c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f2807b7c7db45848fe6184c9134fc21'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:20.939546 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 827/1639] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:20 2022] POST /identity/v3/users => generated 309 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:20.974389 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d09445-5bd4-482c-b2d3-22239465395b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:20.974524 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 813/1640] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:36:20 2022] DELETE /identity/v3/users/7f2807b7c7db45848fe6184c9134fc21 => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:20.974609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a4d09445-5bd4-482c-b2d3-22239465395b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:20.974898 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aca457-6deb-42fd-b78e-42359d818e84 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:20.975189 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aca457-6deb-42fd-b78e-42359d818e84 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:20.975528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aca457-6deb-42fd-b78e-42359d818e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:20.991998 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d09445-5bd4-482c-b2d3-22239465395b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:21.002427 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-41aca457-6deb-42fd-b78e-42359d818e84 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:21.003781 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-41aca457-6deb-42fd-b78e-42359d818e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:21.011910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:21.073219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 814/1641] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:21.087259 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aca457-6deb-42fd-b78e-42359d818e84 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:21.088573 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-41aca457-6deb-42fd-b78e-42359d818e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:21.104213 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086c3ce-4b73-4e21-b960-234351070231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:21.105340 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086c3ce-4b73-4e21-b960-234351070231 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:21.106006 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086c3ce-4b73-4e21-b960-234351070231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:21.106463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086c3ce-4b73-4e21-b960-234351070231 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:21.108843 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086c3ce-4b73-4e21-b960-234351070231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:21.110839 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086c3ce-4b73-4e21-b960-234351070231 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:21.126288 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 828/1642] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:36:21.138437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086c3ce-4b73-4e21-b960-234351070231 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:21.151613 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086c3ce-4b73-4e21-b960-234351070231 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:21.185473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aca457-6deb-42fd-b78e-42359d818e84 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:21.191228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56197e54-9bb9-465c-8115-f7c45709aca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:21.191957 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56197e54-9bb9-465c-8115-f7c45709aca1 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:21.192313 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56197e54-9bb9-465c-8115-f7c45709aca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:21.192759 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56197e54-9bb9-465c-8115-f7c45709aca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:21.196264 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e018bc-1082-4823-a356-a05e8615c332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:21.199685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e018bc-1082-4823-a356-a05e8615c332 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:21.200751 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e018bc-1082-4823-a356-a05e8615c332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:21.204161 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56197e54-9bb9-465c-8115-f7c45709aca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:21.204615 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e018bc-1082-4823-a356-a05e8615c332 admin admin] PATH_INFO: `/v3/projects/e5e8ea57164d47868456cc09e2e4e415/users/c6c7573c169b4415ac4cd2153e89d3b1/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:21.206072 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56197e54-9bb9-465c-8115-f7c45709aca1 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:21.218655 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 815/1643] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:21.236005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:21.243126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56197e54-9bb9-465c-8115-f7c45709aca1 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:21.244631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-56197e54-9bb9-465c-8115-f7c45709aca1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:21.258442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e018bc-1082-4823-a356-a05e8615c332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e8ea57164d47868456cc09e2e4e415, user_id=c6c7573c169b4415ac4cd2153e89d3b1, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:21.268348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e018bc-1082-4823-a356-a05e8615c332 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:21.275590 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56197e54-9bb9-465c-8115-f7c45709aca1 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:21.277364 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9be06-4731-470c-bab1-7ff721c06b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:21.278012 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9be06-4731-470c-bab1-7ff721c06b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:21.278356 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9be06-4731-470c-bab1-7ff721c06b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:21.278677 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9be06-4731-470c-bab1-7ff721c06b57 admin admin] PATH_INFO: `/v3/users/5fac876a606247808751df1ec5efcc9f` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:21.279830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9be06-4731-470c-bab1-7ff721c06b57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fac876a606247808751df1ec5efcc9f)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:21.281989 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9be06-4731-470c-bab1-7ff721c06b57 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:21.417342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 829/1644] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:21 2022] PUT /identity/v3/projects/e5e8ea57164d47868456cc09e2e4e415/users/c6c7573c169b4415ac4cd2153e89d3b1/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:21.422337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e018bc-1082-4823-a356-a05e8615c332 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:21.529473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-31e9be06-4731-470c-bab1-7ff721c06b57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fac876a606247808751df1ec5efcc9f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:21.586319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e018bc-1082-4823-a356-a05e8615c332 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:21.595350 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf056e-2913-4b86-9aa3-fe835d4da481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:21.596252 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf056e-2913-4b86-9aa3-fe835d4da481 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:21.596635 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf056e-2913-4b86-9aa3-fe835d4da481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:21.597035 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf056e-2913-4b86-9aa3-fe835d4da481 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:21.602056 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf056e-2913-4b86-9aa3-fe835d4da481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:21.607816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf056e-2913-4b86-9aa3-fe835d4da481 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:21.628182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-31e9be06-4731-470c-bab1-7ff721c06b57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fac876a606247808751df1ec5efcc9f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:21.628872 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-31e9be06-4731-470c-bab1-7ff721c06b57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fac876a606247808751df1ec5efcc9f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:21.629574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-31e9be06-4731-470c-bab1-7ff721c06b57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fac876a606247808751df1ec5efcc9f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:21.651622 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-31e9be06-4731-470c-bab1-7ff721c06b57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fac876a606247808751df1ec5efcc9f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:21.659731 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 830/1645] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:36:21.664009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf056e-2913-4b86-9aa3-fe835d4da481 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:21.670281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-dfcf056e-2913-4b86-9aa3-fe835d4da481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:21.689229 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 816/1646] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:36:21 2022] DELETE /identity/v3/users/5fac876a606247808751df1ec5efcc9f => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:21.699089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f548708-e6ab-4aad-8d30-aa6effe2c0cc None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:21.699089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f548708-e6ab-4aad-8d30-aa6effe2c0cc None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:21.699089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f548708-e6ab-4aad-8d30-aa6effe2c0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:21.707564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf056e-2913-4b86-9aa3-fe835d4da481 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:21.727130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-5f548708-e6ab-4aad-8d30-aa6effe2c0cc None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:21.728421 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5f548708-e6ab-4aad-8d30-aa6effe2c0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:21.847999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 817/1647] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:21.885703 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c3b8b-6f48-47ea-b25a-bca77b5c6ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:21.887189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c3b8b-6f48-47ea-b25a-bca77b5c6ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:21.887712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c3b8b-6f48-47ea-b25a-bca77b5c6ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:21.888190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c3b8b-6f48-47ea-b25a-bca77b5c6ab2 admin admin] PATH_INFO: `/v3/projects/e5e8ea57164d47868456cc09e2e4e415/users/c6c7573c169b4415ac4cd2153e89d3b1/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:21.892003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c3b8b-6f48-47ea-b25a-bca77b5c6ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e8ea57164d47868456cc09e2e4e415, user_id=c6c7573c169b4415ac4cd2153e89d3b1, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:21.895273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c3b8b-6f48-47ea-b25a-bca77b5c6ab2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:21.914857 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 831/1648] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:21 2022] PUT /identity/v3/projects/e5e8ea57164d47868456cc09e2e4e415/users/c6c7573c169b4415ac4cd2153e89d3b1/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:36:22.132478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f548708-e6ab-4aad-8d30-aa6effe2c0cc None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:22.134534 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5f548708-e6ab-4aad-8d30-aa6effe2c0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:22.233011 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f548708-e6ab-4aad-8d30-aa6effe2c0cc None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:22.297570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296cc06-b106-44a1-9e7b-744e529adc24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:22.298886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296cc06-b106-44a1-9e7b-744e529adc24 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:22.299364 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296cc06-b106-44a1-9e7b-744e529adc24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:22.299795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296cc06-b106-44a1-9e7b-744e529adc24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:22.403347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296cc06-b106-44a1-9e7b-744e529adc24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:22.406024 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296cc06-b106-44a1-9e7b-744e529adc24 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:22.420355 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 818/1649] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:22.810872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c3b8b-6f48-47ea-b25a-bca77b5c6ab2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:22.815494 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d97c3b8b-6f48-47ea-b25a-bca77b5c6ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:22.835063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c3b8b-6f48-47ea-b25a-bca77b5c6ab2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:22.848970 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4811789-2342-4986-9e67-d2d17bd59692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:22.850062 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4811789-2342-4986-9e67-d2d17bd59692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:22.850536 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4811789-2342-4986-9e67-d2d17bd59692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:22.850962 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4811789-2342-4986-9e67-d2d17bd59692 admin admin] PATH_INFO: `/v3/projects/dc87da0661ec4a6caef7e46c13a39482` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:22.852587 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4811789-2342-4986-9e67-d2d17bd59692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc87da0661ec4a6caef7e46c13a39482)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:22.853826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: WARNING py.warnings [None req-a4811789-2342-4986-9e67-d2d17bd59692 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 10:36:22.853826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: warnings.warn(msg) Sep 19 10:36:22.853826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: Sep 19 10:36:22.857448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4811789-2342-4986-9e67-d2d17bd59692 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:22.903587 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:22.907760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:23.072113 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a4811789-2342-4986-9e67-d2d17bd59692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc87da0661ec4a6caef7e46c13a39482', 'request_id': 'req-a4811789-2342-4986-9e67-d2d17bd59692'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:23.104323 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a4811789-2342-4986-9e67-d2d17bd59692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc87da0661ec4a6caef7e46c13a39482', 'request_id': 'req-a4811789-2342-4986-9e67-d2d17bd59692'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:23.144655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a4811789-2342-4986-9e67-d2d17bd59692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc87da0661ec4a6caef7e46c13a39482', 'request_id': 'req-a4811789-2342-4986-9e67-d2d17bd59692'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:23.243565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a4811789-2342-4986-9e67-d2d17bd59692 admin admin] The token cache is being invalidate because project dc87da0661ec4a6caef7e46c13a39482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:36:23.244304 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a4811789-2342-4986-9e67-d2d17bd59692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:23.245132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a4811789-2342-4986-9e67-d2d17bd59692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:23.251827 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 832/1650] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:36:22 2022] DELETE /identity/v3/projects/dc87da0661ec4a6caef7e46c13a39482 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:23.256655 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1206773-3792-42f1-ab5b-215984ec1bc8 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:23.256941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1206773-3792-42f1-ab5b-215984ec1bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:23.257245 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1206773-3792-42f1-ab5b-215984ec1bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:23.298025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-d1206773-3792-42f1-ab5b-215984ec1bc8 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:23.306458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d1206773-3792-42f1-ab5b-215984ec1bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:23.411729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 819/1651] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:23.412306 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a36b62-c09f-4422-9460-edd9c0b08f6a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:23.420951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a36b62-c09f-4422-9460-edd9c0b08f6a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:23.421324 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a36b62-c09f-4422-9460-edd9c0b08f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:23.468370 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1206773-3792-42f1-ab5b-215984ec1bc8 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:23.469974 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d1206773-3792-42f1-ab5b-215984ec1bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:23.485882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-93a36b62-c09f-4422-9460-edd9c0b08f6a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:23.490075 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-93a36b62-c09f-4422-9460-edd9c0b08f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:23.514385 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 833/1652] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:23.515462 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a36b62-c09f-4422-9460-edd9c0b08f6a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:23.516658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1206773-3792-42f1-ab5b-215984ec1bc8 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:23.534980 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855eba9-7abc-4467-8d26-70501bcd8f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:23.535845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855eba9-7abc-4467-8d26-70501bcd8f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:23.536303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855eba9-7abc-4467-8d26-70501bcd8f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:23.536901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855eba9-7abc-4467-8d26-70501bcd8f9c admin admin] PATH_INFO: `/v3/projects/ff0da561855b48c7908860c72039c112` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:23.540473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9855eba9-7abc-4467-8d26-70501bcd8f9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff0da561855b48c7908860c72039c112)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:23.545713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9855eba9-7abc-4467-8d26-70501bcd8f9c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:23.547113 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a36b62-c09f-4422-9460-edd9c0b08f6a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:23.563874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426034e-7e41-4c70-bc19-8091fe40a05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:23.565900 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426034e-7e41-4c70-bc19-8091fe40a05c admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:23.566357 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426034e-7e41-4c70-bc19-8091fe40a05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:23.567159 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426034e-7e41-4c70-bc19-8091fe40a05c admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:23.568630 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426034e-7e41-4c70-bc19-8091fe40a05c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:36:23.569428 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426034e-7e41-4c70-bc19-8091fe40a05c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:23.573963 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426034e-7e41-4c70-bc19-8091fe40a05c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:23.607304 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 834/1653] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:36:23.611350 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426034e-7e41-4c70-bc19-8091fe40a05c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:23.625456 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426034e-7e41-4c70-bc19-8091fe40a05c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:23.630882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3de77-a319-41b1-8215-5de862f3a79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:23.631565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3de77-a319-41b1-8215-5de862f3a79d admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:23.631896 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3de77-a319-41b1-8215-5de862f3a79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:23.632173 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3de77-a319-41b1-8215-5de862f3a79d admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:23.632817 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3de77-a319-41b1-8215-5de862f3a79d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:23.635198 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3de77-a319-41b1-8215-5de862f3a79d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:23.739414 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 835/1654] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:23 2022] POST /identity/v3/projects => generated 355 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:23.743195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3de77-a319-41b1-8215-5de862f3a79d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:23.807198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-9855eba9-7abc-4467-8d26-70501bcd8f9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff0da561855b48c7908860c72039c112', 'request_id': 'req-9855eba9-7abc-4467-8d26-70501bcd8f9c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:23.832821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-9855eba9-7abc-4467-8d26-70501bcd8f9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff0da561855b48c7908860c72039c112', 'request_id': 'req-9855eba9-7abc-4467-8d26-70501bcd8f9c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:23.841844 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-9855eba9-7abc-4467-8d26-70501bcd8f9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff0da561855b48c7908860c72039c112', 'request_id': 'req-9855eba9-7abc-4467-8d26-70501bcd8f9c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:23.888027 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3de77-a319-41b1-8215-5de862f3a79d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:23.957147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-9855eba9-7abc-4467-8d26-70501bcd8f9c admin admin] The token cache is being invalidate because project ff0da561855b48c7908860c72039c112 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:36:23.958181 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-9855eba9-7abc-4467-8d26-70501bcd8f9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:23.960111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-9855eba9-7abc-4467-8d26-70501bcd8f9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:23.962577 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 820/1655] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:36:23 2022] DELETE /identity/v3/projects/ff0da561855b48c7908860c72039c112 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:23.968687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855eba9-7abc-4467-8d26-70501bcd8f9c admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:23.970993 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-9855eba9-7abc-4467-8d26-70501bcd8f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:24.026736 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2ea7e-2242-46af-8d01-6464a844ae9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:24.027865 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2ea7e-2242-46af-8d01-6464a844ae9c admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:24.029258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2ea7e-2242-46af-8d01-6464a844ae9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:24.029668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2ea7e-2242-46af-8d01-6464a844ae9c admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:24.030478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2ea7e-2242-46af-8d01-6464a844ae9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:24.033292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2ea7e-2242-46af-8d01-6464a844ae9c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:24.107812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855eba9-7abc-4467-8d26-70501bcd8f9c admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:24.122677 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164ce23-770d-41d7-b4d0-66b1e11a50d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:24.123443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164ce23-770d-41d7-b4d0-66b1e11a50d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:24.123815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164ce23-770d-41d7-b4d0-66b1e11a50d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:24.124179 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164ce23-770d-41d7-b4d0-66b1e11a50d8 admin admin] PATH_INFO: `/v3/users/1e44d6f9bbd340a89f8574039194f3c2` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:24.126030 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164ce23-770d-41d7-b4d0-66b1e11a50d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e44d6f9bbd340a89f8574039194f3c2)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:24.130859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164ce23-770d-41d7-b4d0-66b1e11a50d8 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:24.180975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 836/1656] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:23 2022] POST /identity/v3/users => generated 278 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:24.186995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2ea7e-2242-46af-8d01-6464a844ae9c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:24.191831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0ed2ea7e-2242-46af-8d01-6464a844ae9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:24.215653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2ea7e-2242-46af-8d01-6464a844ae9c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:24.221807 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54c5d2-6bf2-4211-8ce5-923e69fbd161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:24.224720 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54c5d2-6bf2-4211-8ce5-923e69fbd161 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:24.224720 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54c5d2-6bf2-4211-8ce5-923e69fbd161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:24.224948 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54c5d2-6bf2-4211-8ce5-923e69fbd161 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:24.230102 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54c5d2-6bf2-4211-8ce5-923e69fbd161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:24.232474 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54c5d2-6bf2-4211-8ce5-923e69fbd161 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:24.247686 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 837/1657] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:24.251602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54c5d2-6bf2-4211-8ce5-923e69fbd161 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:24.267720 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54c5d2-6bf2-4211-8ce5-923e69fbd161 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:24.273883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e355b3-2e93-474d-8789-a5a117cb598c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:24.274856 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e355b3-2e93-474d-8789-a5a117cb598c admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:24.280828 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e355b3-2e93-474d-8789-a5a117cb598c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:24.280828 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e355b3-2e93-474d-8789-a5a117cb598c admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:24.280828 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e355b3-2e93-474d-8789-a5a117cb598c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:24.280828 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e355b3-2e93-474d-8789-a5a117cb598c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:24.294744 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 838/1658] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:24.299493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e355b3-2e93-474d-8789-a5a117cb598c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:24.316244 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e355b3-2e93-474d-8789-a5a117cb598c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:24.342419 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eaa6f5-783f-4cfe-9cf1-eeafddd48bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:24.343062 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eaa6f5-783f-4cfe-9cf1-eeafddd48bcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:24.343343 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eaa6f5-783f-4cfe-9cf1-eeafddd48bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:24.343803 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eaa6f5-783f-4cfe-9cf1-eeafddd48bcd admin admin] PATH_INFO: `/v3/projects/e462cb6b51f04221a55a34cd8ecece46/users/6e439fb618af4404b9fbafbad6b996f9/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:24.372353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:24.385880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eaa6f5-783f-4cfe-9cf1-eeafddd48bcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e462cb6b51f04221a55a34cd8ecece46, user_id=6e439fb618af4404b9fbafbad6b996f9, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:24.403966 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eaa6f5-783f-4cfe-9cf1-eeafddd48bcd admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:24.416610 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-b164ce23-770d-41d7-b4d0-66b1e11a50d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e44d6f9bbd340a89f8574039194f3c2'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:24.508527 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-b164ce23-770d-41d7-b4d0-66b1e11a50d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e44d6f9bbd340a89f8574039194f3c2'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:24.509309 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-b164ce23-770d-41d7-b4d0-66b1e11a50d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e44d6f9bbd340a89f8574039194f3c2'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:24.518422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-b164ce23-770d-41d7-b4d0-66b1e11a50d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e44d6f9bbd340a89f8574039194f3c2'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:24.519283 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 839/1659] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:24 2022] PUT /identity/v3/projects/e462cb6b51f04221a55a34cd8ecece46/users/6e439fb618af4404b9fbafbad6b996f9/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:36:24.521976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-b164ce23-770d-41d7-b4d0-66b1e11a50d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e44d6f9bbd340a89f8574039194f3c2'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:24.542272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d566c-2920-47c1-b0c2-b996166ae5f7 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:24.542735 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d566c-2920-47c1-b0c2-b996166ae5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:24.547830 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d566c-2920-47c1-b0c2-b996166ae5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:24.576458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 821/1660] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:36:23 2022] DELETE /identity/v3/users/1e44d6f9bbd340a89f8574039194f3c2 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:24.593038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f09d566c-2920-47c1-b0c2-b996166ae5f7 None None] MFA Rules not processed for user `6e439fb618af4404b9fbafbad6b996f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:24.594717 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f09d566c-2920-47c1-b0c2-b996166ae5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:24.663054 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164ce23-770d-41d7-b4d0-66b1e11a50d8 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:24.663054 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b164ce23-770d-41d7-b4d0-66b1e11a50d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:24.787905 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 840/1661] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:36:24.842902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d566c-2920-47c1-b0c2-b996166ae5f7 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:24.844530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f09d566c-2920-47c1-b0c2-b996166ae5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:24.847014 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164ce23-770d-41d7-b4d0-66b1e11a50d8 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:24.858867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08c114-a2c5-4ddd-8739-7aa8d7cfcaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:24.859892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08c114-a2c5-4ddd-8739-7aa8d7cfcaca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:24.860884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d566c-2920-47c1-b0c2-b996166ae5f7 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:24.861406 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08c114-a2c5-4ddd-8739-7aa8d7cfcaca admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:24.861912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08c114-a2c5-4ddd-8739-7aa8d7cfcaca admin admin] PATH_INFO: `/v3/users/01008443277843d5b61807b64bdd8c72` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:24.866538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb1406-4a3f-4684-ab80-9e9b5ee8864e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:24.867181 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb1406-4a3f-4684-ab80-9e9b5ee8864e admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:24.867460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb1406-4a3f-4684-ab80-9e9b5ee8864e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:24.867734 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb1406-4a3f-4684-ab80-9e9b5ee8864e admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:24.868383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb1406-4a3f-4684-ab80-9e9b5ee8864e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:24.870836 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb1406-4a3f-4684-ab80-9e9b5ee8864e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:24.872096 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea08c114-a2c5-4ddd-8739-7aa8d7cfcaca admin admin] RBAC: Authorizing `identity:delete_user(user_id=01008443277843d5b61807b64bdd8c72)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:24.878216 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea08c114-a2c5-4ddd-8739-7aa8d7cfcaca admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:25.063031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 841/1662] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:24 2022] POST /identity/v3/projects => generated 355 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:25.068415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb1406-4a3f-4684-ab80-9e9b5ee8864e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:25.245691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ea08c114-a2c5-4ddd-8739-7aa8d7cfcaca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01008443277843d5b61807b64bdd8c72'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:25.260526 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb1406-4a3f-4684-ab80-9e9b5ee8864e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:25.275540 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81a3fe-da39-4ffb-ab11-510ca8b4c003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:25.276690 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81a3fe-da39-4ffb-ab11-510ca8b4c003 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:25.277079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81a3fe-da39-4ffb-ab11-510ca8b4c003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:25.277268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81a3fe-da39-4ffb-ab11-510ca8b4c003 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:25.280212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81a3fe-da39-4ffb-ab11-510ca8b4c003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:25.286697 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81a3fe-da39-4ffb-ab11-510ca8b4c003 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:25.342231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ea08c114-a2c5-4ddd-8739-7aa8d7cfcaca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01008443277843d5b61807b64bdd8c72'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:25.342944 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ea08c114-a2c5-4ddd-8739-7aa8d7cfcaca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01008443277843d5b61807b64bdd8c72'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:25.343790 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ea08c114-a2c5-4ddd-8739-7aa8d7cfcaca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01008443277843d5b61807b64bdd8c72'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:25.371168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ea08c114-a2c5-4ddd-8739-7aa8d7cfcaca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01008443277843d5b61807b64bdd8c72'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:25.419887 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 842/1663] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:25 2022] POST /identity/v3/users => generated 284 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:25.430562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81a3fe-da39-4ffb-ab11-510ca8b4c003 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:25.430562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6e81a3fe-da39-4ffb-ab11-510ca8b4c003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:25.438361 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 822/1664] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:36:24 2022] DELETE /identity/v3/users/01008443277843d5b61807b64bdd8c72 => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:36:25.443827 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d51a26-226d-44fe-9f50-0044adfbca20 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:25.444381 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d51a26-226d-44fe-9f50-0044adfbca20 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:25.444874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d51a26-226d-44fe-9f50-0044adfbca20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:25.470354 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81a3fe-da39-4ffb-ab11-510ca8b4c003 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:25.520509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-e4d51a26-226d-44fe-9f50-0044adfbca20 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:25.528268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e4d51a26-226d-44fe-9f50-0044adfbca20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:25.548874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:25.689489 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed0235-a1d8-4029-a042-47aa7da6ff98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:25.690379 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed0235-a1d8-4029-a042-47aa7da6ff98 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:25.691197 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed0235-a1d8-4029-a042-47aa7da6ff98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:25.692492 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed0235-a1d8-4029-a042-47aa7da6ff98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:25.695084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed0235-a1d8-4029-a042-47aa7da6ff98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:25.698748 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed0235-a1d8-4029-a042-47aa7da6ff98 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:25.753247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 823/1665] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:36:25.753247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6440d1-218d-419a-8198-0e6358625758 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:25.753247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6440d1-218d-419a-8198-0e6358625758 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:25.753247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6440d1-218d-419a-8198-0e6358625758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:25.760094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 843/1666] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:25.763062 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed0235-a1d8-4029-a042-47aa7da6ff98 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:25.805683 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed0235-a1d8-4029-a042-47aa7da6ff98 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:25.827826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8c865-b921-4c62-88ea-44b8b85eeb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:25.843009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8c865-b921-4c62-88ea-44b8b85eeb33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:25.846826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8c865-b921-4c62-88ea-44b8b85eeb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:25.848910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8c865-b921-4c62-88ea-44b8b85eeb33 admin admin] PATH_INFO: `/v3/projects/8e1f2d4b68af47598f61ce902b3d8680/users/b014e1df09434b728e6b76f8fcd38f16/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:25.872900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-2b6440d1-218d-419a-8198-0e6358625758 None None] MFA Rules not processed for user `c6c7573c169b4415ac4cd2153e89d3b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:25.874292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-2b6440d1-218d-419a-8198-0e6358625758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:25.891692 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8c865-b921-4c62-88ea-44b8b85eeb33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e1f2d4b68af47598f61ce902b3d8680, user_id=b014e1df09434b728e6b76f8fcd38f16, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:25.896719 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8c865-b921-4c62-88ea-44b8b85eeb33 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:26.001886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 844/1667] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:25 2022] PUT /identity/v3/projects/8e1f2d4b68af47598f61ce902b3d8680/users/b014e1df09434b728e6b76f8fcd38f16/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:26.003036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8c865-b921-4c62-88ea-44b8b85eeb33 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:26.006834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f7e8c865-b921-4c62-88ea-44b8b85eeb33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:26.038163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 824/1668] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:25 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:26.047705 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6440d1-218d-419a-8198-0e6358625758 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:26.075775 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6440d1-218d-419a-8198-0e6358625758 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:26.201854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8c865-b921-4c62-88ea-44b8b85eeb33 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:26.223210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f382d8-cb02-4b22-bec6-63ec6df8cd95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:26.223846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f382d8-cb02-4b22-bec6-63ec6df8cd95 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:26.224123 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f382d8-cb02-4b22-bec6-63ec6df8cd95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:26.224393 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f382d8-cb02-4b22-bec6-63ec6df8cd95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:26.232431 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e17e5-a1be-43ac-9c86-9883454036a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:26.233393 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e17e5-a1be-43ac-9c86-9883454036a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:26.233675 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e17e5-a1be-43ac-9c86-9883454036a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:26.234017 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e17e5-a1be-43ac-9c86-9883454036a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:26.234590 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e17e5-a1be-43ac-9c86-9883454036a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:26.242325 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e17e5-a1be-43ac-9c86-9883454036a4 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:26.259962 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f382d8-cb02-4b22-bec6-63ec6df8cd95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:26.261928 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f382d8-cb02-4b22-bec6-63ec6df8cd95 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:26.292980 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 825/1669] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:26.292980 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e17e5-a1be-43ac-9c86-9883454036a4 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:26.300725 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 845/1670] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:26.318094 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e17e5-a1be-43ac-9c86-9883454036a4 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:26.339909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455842aa-cc78-402b-b008-c8adc796aaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:26.341223 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455842aa-cc78-402b-b008-c8adc796aaac admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:26.341506 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455842aa-cc78-402b-b008-c8adc796aaac admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:26.341786 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455842aa-cc78-402b-b008-c8adc796aaac admin admin] PATH_INFO: `/v3/projects/8e1f2d4b68af47598f61ce902b3d8680/users/b014e1df09434b728e6b76f8fcd38f16/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:26.343854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455842aa-cc78-402b-b008-c8adc796aaac admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e1f2d4b68af47598f61ce902b3d8680, user_id=b014e1df09434b728e6b76f8fcd38f16, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:26.355404 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455842aa-cc78-402b-b008-c8adc796aaac admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:26.398480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 826/1671] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:26 2022] PUT /identity/v3/projects/8e1f2d4b68af47598f61ce902b3d8680/users/b014e1df09434b728e6b76f8fcd38f16/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:26.398719 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c4241-012f-4c9a-b9df-3342afdec7a8 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:26.398719 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c4241-012f-4c9a-b9df-3342afdec7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:26.398719 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c4241-012f-4c9a-b9df-3342afdec7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:26.401139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455842aa-cc78-402b-b008-c8adc796aaac admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:26.403890 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-455842aa-cc78-402b-b008-c8adc796aaac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:26.484631 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c43c4241-012f-4c9a-b9df-3342afdec7a8 None None] MFA Rules not processed for user `b014e1df09434b728e6b76f8fcd38f16`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:26.490519 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c43c4241-012f-4c9a-b9df-3342afdec7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:26.556222 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:26.585558 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455842aa-cc78-402b-b008-c8adc796aaac admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:26.599039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2236d-f55d-41fe-b251-c32a4d6c9a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:26.602972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2236d-f55d-41fe-b251-c32a4d6c9a7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:26.602972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2236d-f55d-41fe-b251-c32a4d6c9a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:26.602972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2236d-f55d-41fe-b251-c32a4d6c9a7c admin admin] PATH_INFO: `/v3/projects/38b637de610d401483dbfa2f42db92b3` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:26.606906 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2236d-f55d-41fe-b251-c32a4d6c9a7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=38b637de610d401483dbfa2f42db92b3)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:26.610322 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2236d-f55d-41fe-b251-c32a4d6c9a7c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:26.677040 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:26.682837 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 846/1672] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:26.696986 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c4241-012f-4c9a-b9df-3342afdec7a8 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:26.698592 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c43c4241-012f-4c9a-b9df-3342afdec7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:26.840828 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2aa2236d-f55d-41fe-b251-c32a4d6c9a7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38b637de610d401483dbfa2f42db92b3', 'request_id': 'req-2aa2236d-f55d-41fe-b251-c32a4d6c9a7c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:26.843345 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c4241-012f-4c9a-b9df-3342afdec7a8 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:26.849696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2aa2236d-f55d-41fe-b251-c32a4d6c9a7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38b637de610d401483dbfa2f42db92b3', 'request_id': 'req-2aa2236d-f55d-41fe-b251-c32a4d6c9a7c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:26.867052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2aa2236d-f55d-41fe-b251-c32a4d6c9a7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38b637de610d401483dbfa2f42db92b3', 'request_id': 'req-2aa2236d-f55d-41fe-b251-c32a4d6c9a7c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:26.928682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772883ba-6f7c-4b20-8c5e-89ab6a272090 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:26.929929 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772883ba-6f7c-4b20-8c5e-89ab6a272090 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:26.930693 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772883ba-6f7c-4b20-8c5e-89ab6a272090 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:26.931202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772883ba-6f7c-4b20-8c5e-89ab6a272090 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:27.016213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2aa2236d-f55d-41fe-b251-c32a4d6c9a7c admin admin] The token cache is being invalidate because project 38b637de610d401483dbfa2f42db92b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:36:27.017023 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2aa2236d-f55d-41fe-b251-c32a4d6c9a7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:27.018242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772883ba-6f7c-4b20-8c5e-89ab6a272090 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:27.018460 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2aa2236d-f55d-41fe-b251-c32a4d6c9a7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:27.020992 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 827/1673] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:36:26 2022] DELETE /identity/v3/projects/38b637de610d401483dbfa2f42db92b3 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:36:27.021571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772883ba-6f7c-4b20-8c5e-89ab6a272090 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:27.038062 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19da440-0ae0-4b8d-8334-c22ab0327a54 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:27.044229 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19da440-0ae0-4b8d-8334-c22ab0327a54 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:27.044229 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19da440-0ae0-4b8d-8334-c22ab0327a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:27.047489 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 847/1674] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:27.096298 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-b19da440-0ae0-4b8d-8334-c22ab0327a54 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:27.097663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b19da440-0ae0-4b8d-8334-c22ab0327a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:27.255367 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 828/1675] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:27.265447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d94ca-78f7-44fe-ab5f-447c6003fec9 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:27.266008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d94ca-78f7-44fe-ab5f-447c6003fec9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:27.266102 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d94ca-78f7-44fe-ab5f-447c6003fec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:27.306397 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-864d94ca-78f7-44fe-ab5f-447c6003fec9 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:27.307873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-864d94ca-78f7-44fe-ab5f-447c6003fec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:27.326765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19da440-0ae0-4b8d-8334-c22ab0327a54 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:27.328348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b19da440-0ae0-4b8d-8334-c22ab0327a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:27.332805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 848/1676] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:27.334169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d94ca-78f7-44fe-ab5f-447c6003fec9 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:27.344781 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d94ca-78f7-44fe-ab5f-447c6003fec9 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:27.350513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19da440-0ae0-4b8d-8334-c22ab0327a54 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:27.356747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbe57d-8582-41b2-bc81-6b69ffdd49ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:27.356747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbe57d-8582-41b2-bc81-6b69ffdd49ec admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:27.356747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbe57d-8582-41b2-bc81-6b69ffdd49ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:27.356747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbe57d-8582-41b2-bc81-6b69ffdd49ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:27.357476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbe57d-8582-41b2-bc81-6b69ffdd49ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:36:27.357476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbe57d-8582-41b2-bc81-6b69ffdd49ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:27.358062 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbe57d-8582-41b2-bc81-6b69ffdd49ec admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:27.363756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c49af8-ae97-4bd8-9889-ccdb947961ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:27.366320 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c49af8-ae97-4bd8-9889-ccdb947961ba service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:27.366693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c49af8-ae97-4bd8-9889-ccdb947961ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:27.367043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c49af8-ae97-4bd8-9889-ccdb947961ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:27.373455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e7c49af8-ae97-4bd8-9889-ccdb947961ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:27.380067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 849/1677] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:36:27.388303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbe57d-8582-41b2-bc81-6b69ffdd49ec admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:27.396542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbe57d-8582-41b2-bc81-6b69ffdd49ec admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:27.412196 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd867959-c8d0-4854-9faf-39fe983acbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:27.413564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd867959-c8d0-4854-9faf-39fe983acbeb admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:27.413564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd867959-c8d0-4854-9faf-39fe983acbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:27.413564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd867959-c8d0-4854-9faf-39fe983acbeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:27.414673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd867959-c8d0-4854-9faf-39fe983acbeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:27.422802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd867959-c8d0-4854-9faf-39fe983acbeb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:27.480836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c49af8-ae97-4bd8-9889-ccdb947961ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:27.487234 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c49af8-ae97-4bd8-9889-ccdb947961ba service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:27.502473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 829/1678] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:27 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:36:27.584227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 850/1679] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:27 2022] POST /identity/v3/projects => generated 387 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:27.585602 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c49af8-ae97-4bd8-9889-ccdb947961ba service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:27.591691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:27.602300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c49af8-ae97-4bd8-9889-ccdb947961ba service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:27.773876 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0cba5-3e61-4279-b659-eb3675ec54d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:27.774773 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0cba5-3e61-4279-b659-eb3675ec54d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:27.775210 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0cba5-3e61-4279-b659-eb3675ec54d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:27.775641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0cba5-3e61-4279-b659-eb3675ec54d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:27.776464 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0cba5-3e61-4279-b659-eb3675ec54d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:27.779220 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0cba5-3e61-4279-b659-eb3675ec54d6 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:27.900601 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 830/1680] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:27 2022] POST /identity/v3/users => generated 294 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:27.904791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd867959-c8d0-4854-9faf-39fe983acbeb admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:27.925535 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd867959-c8d0-4854-9faf-39fe983acbeb admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:27.939196 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae89d2a-b43a-44a8-939a-9b28467e554e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:27.939845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae89d2a-b43a-44a8-939a-9b28467e554e admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:27.940417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae89d2a-b43a-44a8-939a-9b28467e554e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:27.941565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae89d2a-b43a-44a8-939a-9b28467e554e admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:27.945045 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae89d2a-b43a-44a8-939a-9b28467e554e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:27.949866 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae89d2a-b43a-44a8-939a-9b28467e554e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:27.976427 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 851/1681] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:27.980529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0cba5-3e61-4279-b659-eb3675ec54d6 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:27.997939 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0cba5-3e61-4279-b659-eb3675ec54d6 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:28.009135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccc4dc-ea17-4799-8995-b573809949e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:28.009873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccc4dc-ea17-4799-8995-b573809949e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:28.010157 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccc4dc-ea17-4799-8995-b573809949e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:28.010432 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccc4dc-ea17-4799-8995-b573809949e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:28.011093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccc4dc-ea17-4799-8995-b573809949e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:28.013079 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccc4dc-ea17-4799-8995-b573809949e3 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:28.077727 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae89d2a-b43a-44a8-939a-9b28467e554e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:28.093357 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae89d2a-b43a-44a8-939a-9b28467e554e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:28.094080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 831/1682] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:28.108924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b46145-e24c-48e3-b954-635439f32495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:28.110222 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b46145-e24c-48e3-b954-635439f32495 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:28.110365 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b46145-e24c-48e3-b954-635439f32495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:28.110653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b46145-e24c-48e3-b954-635439f32495 admin admin] PATH_INFO: `/v3/projects/ff7bba98ba344851a48ae64840a7106c/users/0a175b8016544995be2ebc976ac058ed/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:28.159486 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b46145-e24c-48e3-b954-635439f32495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff7bba98ba344851a48ae64840a7106c, user_id=0a175b8016544995be2ebc976ac058ed, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:28.162293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b46145-e24c-48e3-b954-635439f32495 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:28.266272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 852/1683] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:28 2022] PUT /identity/v3/projects/ff7bba98ba344851a48ae64840a7106c/users/0a175b8016544995be2ebc976ac058ed/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:36:28.277561 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bbbff-b4f8-41a4-838d-766657c40b4a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:28.280707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bbbff-b4f8-41a4-838d-766657c40b4a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:28.280707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bbbff-b4f8-41a4-838d-766657c40b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:28.287956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:28.323493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-738bbbff-b4f8-41a4-838d-766657c40b4a None None] MFA Rules not processed for user `0a175b8016544995be2ebc976ac058ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:28.325491 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-738bbbff-b4f8-41a4-838d-766657c40b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:28.438170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 832/1684] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:28.438708 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b46145-e24c-48e3-b954-635439f32495 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:28.528496 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b46145-e24c-48e3-b954-635439f32495 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:28.533682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09585ef7-b158-4d52-a208-8a264ea55bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:28.534716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09585ef7-b158-4d52-a208-8a264ea55bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:28.535247 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09585ef7-b158-4d52-a208-8a264ea55bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:28.535734 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09585ef7-b158-4d52-a208-8a264ea55bf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:28.536752 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09585ef7-b158-4d52-a208-8a264ea55bf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:28.540462 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09585ef7-b158-4d52-a208-8a264ea55bf0 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:28.634149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:28.694576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 853/1685] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:28 2022] POST /identity/v3/projects => generated 385 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:36:28.696837 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738bbbff-b4f8-41a4-838d-766657c40b4a None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:28.710079 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738bbbff-b4f8-41a4-838d-766657c40b4a None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:28.822855 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bca07-5f1d-4985-a3e3-6d687c824384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:28.826362 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bca07-5f1d-4985-a3e3-6d687c824384 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:28.826725 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bca07-5f1d-4985-a3e3-6d687c824384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:28.827056 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bca07-5f1d-4985-a3e3-6d687c824384 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:28.828020 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330bca07-5f1d-4985-a3e3-6d687c824384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:28.830362 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330bca07-5f1d-4985-a3e3-6d687c824384 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:28.866459 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983d440-5f07-4325-a6e4-e1a6eb1a1340 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:28.866744 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983d440-5f07-4325-a6e4-e1a6eb1a1340 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:28.867026 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983d440-5f07-4325-a6e4-e1a6eb1a1340 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:28.868542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 854/1686] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 19 10:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:36:28.884920 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713e375-5ffa-4d1c-968f-26b7416042ed None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:28.886702 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713e375-5ffa-4d1c-968f-26b7416042ed None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:28.893232 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713e375-5ffa-4d1c-968f-26b7416042ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:28.930997 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 833/1687] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:28 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:28.935406 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4713e375-5ffa-4d1c-968f-26b7416042ed None None] MFA Rules not processed for user `ce0ef47a60f54ca1853fd267e308c58b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:28.936045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bca07-5f1d-4985-a3e3-6d687c824384 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:28.937232 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4713e375-5ffa-4d1c-968f-26b7416042ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:28.971537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bca07-5f1d-4985-a3e3-6d687c824384 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:28.985447 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211bbf0-4dd9-4a63-940d-9c467184031c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:28.986140 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211bbf0-4dd9-4a63-940d-9c467184031c admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:28.988116 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211bbf0-4dd9-4a63-940d-9c467184031c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:28.990286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211bbf0-4dd9-4a63-940d-9c467184031c admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:28.991484 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211bbf0-4dd9-4a63-940d-9c467184031c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:29.002262 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211bbf0-4dd9-4a63-940d-9c467184031c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:29.021891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 855/1688] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 19 10:36:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:29.045209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 834/1689] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:29.050098 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713e375-5ffa-4d1c-968f-26b7416042ed None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:29.073020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713e375-5ffa-4d1c-968f-26b7416042ed None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:29.083152 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bb099-9898-4f9e-858b-3c180d22082c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:29.084214 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bb099-9898-4f9e-858b-3c180d22082c admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:29.084695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bb099-9898-4f9e-858b-3c180d22082c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:29.085237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bb099-9898-4f9e-858b-3c180d22082c admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:29.086507 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bb099-9898-4f9e-858b-3c180d22082c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:29.089053 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bb099-9898-4f9e-858b-3c180d22082c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:29.110364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 856/1690] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:29.110720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211bbf0-4dd9-4a63-940d-9c467184031c admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:29.126869 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211bbf0-4dd9-4a63-940d-9c467184031c admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:29.137860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30dc6fe-100b-483e-9834-a671e6af7a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:29.139041 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30dc6fe-100b-483e-9834-a671e6af7a95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:29.139328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30dc6fe-100b-483e-9834-a671e6af7a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:29.139651 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30dc6fe-100b-483e-9834-a671e6af7a95 admin admin] PATH_INFO: `/v3/projects/0d9786b577494ccf859f6585f119a9bc/users/079f07a5d8cd43ab9b26441e93a8c9a3/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:29.195171 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30dc6fe-100b-483e-9834-a671e6af7a95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d9786b577494ccf859f6585f119a9bc, user_id=079f07a5d8cd43ab9b26441e93a8c9a3, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:29.206768 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30dc6fe-100b-483e-9834-a671e6af7a95 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:29.293257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bb099-9898-4f9e-858b-3c180d22082c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:29.295702 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0a8bb099-9898-4f9e-858b-3c180d22082c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:29.318966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 835/1691] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:29 2022] PUT /identity/v3/projects/0d9786b577494ccf859f6585f119a9bc/users/079f07a5d8cd43ab9b26441e93a8c9a3/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:36:29.320582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b951e3-7660-4383-ae01-ea31f2b6a5d1 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:29.321146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b951e3-7660-4383-ae01-ea31f2b6a5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:29.321562 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b951e3-7660-4383-ae01-ea31f2b6a5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:29.350962 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bb099-9898-4f9e-858b-3c180d22082c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:29.419395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-f6b951e3-7660-4383-ae01-ea31f2b6a5d1 None None] MFA Rules not processed for user `8a059ba74d744a12a4694513f8ca5789`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:29.420726 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-f6b951e3-7660-4383-ae01-ea31f2b6a5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:29.480080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6af73-c501-4034-8437-4c6b75510af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:29.480712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6af73-c501-4034-8437-4c6b75510af4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:29.481239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6af73-c501-4034-8437-4c6b75510af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:29.483268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6af73-c501-4034-8437-4c6b75510af4 admin admin] PATH_INFO: `/v3/projects/2998988ae7ae429ca62921ac188de9b9` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:29.485532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf6af73-c501-4034-8437-4c6b75510af4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2998988ae7ae429ca62921ac188de9b9)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:29.488927 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf6af73-c501-4034-8437-4c6b75510af4 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:29.605974 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 836/1692] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:29.607432 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b951e3-7660-4383-ae01-ea31f2b6a5d1 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:29.611401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-f6b951e3-7660-4383-ae01-ea31f2b6a5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:29.683015 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fcf6af73-c501-4034-8437-4c6b75510af4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2998988ae7ae429ca62921ac188de9b9', 'request_id': 'req-fcf6af73-c501-4034-8437-4c6b75510af4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:29.696092 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fcf6af73-c501-4034-8437-4c6b75510af4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2998988ae7ae429ca62921ac188de9b9', 'request_id': 'req-fcf6af73-c501-4034-8437-4c6b75510af4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:29.709186 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fcf6af73-c501-4034-8437-4c6b75510af4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2998988ae7ae429ca62921ac188de9b9', 'request_id': 'req-fcf6af73-c501-4034-8437-4c6b75510af4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:29.748539 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:29.763463 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b951e3-7660-4383-ae01-ea31f2b6a5d1 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:29.824904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:29.826226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fcf6af73-c501-4034-8437-4c6b75510af4 admin admin] The token cache is being invalidate because project 2998988ae7ae429ca62921ac188de9b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:36:29.826526 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fcf6af73-c501-4034-8437-4c6b75510af4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:29.827402 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fcf6af73-c501-4034-8437-4c6b75510af4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:29.831787 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 857/1693] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:36:29 2022] DELETE /identity/v3/projects/2998988ae7ae429ca62921ac188de9b9 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:29.833652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c5c44-a243-4b7e-b571-2b9eb8a2ffc5 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:29.833945 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c5c44-a243-4b7e-b571-2b9eb8a2ffc5 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:29.834226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c5c44-a243-4b7e-b571-2b9eb8a2ffc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:29.916251 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5b3c5c44-a243-4b7e-b571-2b9eb8a2ffc5 None None] MFA Rules not processed for user `079f07a5d8cd43ab9b26441e93a8c9a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:29.927986 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5b3c5c44-a243-4b7e-b571-2b9eb8a2ffc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:30.003897 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daaa71-5310-49be-a255-ddd24a9e26b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0ef47a60f54ca1853fd267e308c58b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:30.005496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daaa71-5310-49be-a255-ddd24a9e26b5 service placement] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:30.005952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daaa71-5310-49be-a255-ddd24a9e26b5 service placement] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:30.007275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daaa71-5310-49be-a255-ddd24a9e26b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:30.015237 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-14daaa71-5310-49be-a255-ddd24a9e26b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:30.043167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14daaa71-5310-49be-a255-ddd24a9e26b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:30.048582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14daaa71-5310-49be-a255-ddd24a9e26b5 service placement] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:30.077541 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 837/1694] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:30.086089 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 858/1695] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:30.100552 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc292610-9b97-4176-b664-789e31fa379a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:30.100849 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc292610-9b97-4176-b664-789e31fa379a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:30.101158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc292610-9b97-4176-b664-789e31fa379a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:30.103793 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daaa71-5310-49be-a255-ddd24a9e26b5 service placement] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:30.108748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-14daaa71-5310-49be-a255-ddd24a9e26b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:30.136104 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-bc292610-9b97-4176-b664-789e31fa379a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:30.143323 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bc292610-9b97-4176-b664-789e31fa379a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:30.325875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 859/1696] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:30.328102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daaa71-5310-49be-a255-ddd24a9e26b5 service placement] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:30.331416 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc292610-9b97-4176-b664-789e31fa379a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:30.339306 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bc292610-9b97-4176-b664-789e31fa379a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:30.347296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc400ed8-4e31-4109-8be6-50fa3524e653 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:30.347625 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc400ed8-4e31-4109-8be6-50fa3524e653 service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:30.347910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc400ed8-4e31-4109-8be6-50fa3524e653 service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:30.348182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc400ed8-4e31-4109-8be6-50fa3524e653 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:30.356331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-fc400ed8-4e31-4109-8be6-50fa3524e653 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:30.374232 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc292610-9b97-4176-b664-789e31fa379a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:30.396552 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aede09f-3cf9-487c-aca1-1e15f9d0ec75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:30.398582 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aede09f-3cf9-487c-aca1-1e15f9d0ec75 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:30.400172 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aede09f-3cf9-487c-aca1-1e15f9d0ec75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:30.401615 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aede09f-3cf9-487c-aca1-1e15f9d0ec75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:30.402439 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aede09f-3cf9-487c-aca1-1e15f9d0ec75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:30.415363 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aede09f-3cf9-487c-aca1-1e15f9d0ec75 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:30.519486 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc400ed8-4e31-4109-8be6-50fa3524e653 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:30.521243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc400ed8-4e31-4109-8be6-50fa3524e653 service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:30.532721 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 838/1697] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:30.534204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb09dcc-88ec-43ab-83a9-8df6fcb0a7b4 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:30.534615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb09dcc-88ec-43ab-83a9-8df6fcb0a7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:30.535022 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb09dcc-88ec-43ab-83a9-8df6fcb0a7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:30.557666 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-1bb09dcc-88ec-43ab-83a9-8df6fcb0a7b4 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:30.569120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 860/1698] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:30 2022] POST /identity/v3/projects => generated 387 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:30.572518 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-1bb09dcc-88ec-43ab-83a9-8df6fcb0a7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:30.573622 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aede09f-3cf9-487c-aca1-1e15f9d0ec75 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:30.717127 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aede09f-3cf9-487c-aca1-1e15f9d0ec75 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:30.723351 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0949d-3357-4538-8620-fe75af2752ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:30.723907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0949d-3357-4538-8620-fe75af2752ac admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:30.724247 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0949d-3357-4538-8620-fe75af2752ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:30.724671 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0949d-3357-4538-8620-fe75af2752ac admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:30.726122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0949d-3357-4538-8620-fe75af2752ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:30.728215 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0949d-3357-4538-8620-fe75af2752ac admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:30.765238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 839/1699] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:30.765238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb09dcc-88ec-43ab-83a9-8df6fcb0a7b4 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:30.765238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-1bb09dcc-88ec-43ab-83a9-8df6fcb0a7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:30.784961 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb09dcc-88ec-43ab-83a9-8df6fcb0a7b4 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:30.791114 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598039cf-59f2-46b5-88cc-9aafb3020da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:30.791972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598039cf-59f2-46b5-88cc-9aafb3020da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:30.792447 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598039cf-59f2-46b5-88cc-9aafb3020da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:30.792680 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598039cf-59f2-46b5-88cc-9aafb3020da6 admin admin] PATH_INFO: `/v3/users/2895914d8bcc4960a646eac3df8d92f3` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:30.794600 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598039cf-59f2-46b5-88cc-9aafb3020da6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2895914d8bcc4960a646eac3df8d92f3)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:30.796971 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598039cf-59f2-46b5-88cc-9aafb3020da6 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:30.879692 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 861/1700] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:30 2022] POST /identity/v3/users => generated 300 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:30.881046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0949d-3357-4538-8620-fe75af2752ac admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:30.902278 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0949d-3357-4538-8620-fe75af2752ac admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:30.909273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694028b-5f9d-47ed-884b-db8b990790c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:30.910102 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694028b-5f9d-47ed-884b-db8b990790c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:30.910646 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694028b-5f9d-47ed-884b-db8b990790c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:30.911032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694028b-5f9d-47ed-884b-db8b990790c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:30.911804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694028b-5f9d-47ed-884b-db8b990790c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:36:30.912216 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694028b-5f9d-47ed-884b-db8b990790c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:30.914682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694028b-5f9d-47ed-884b-db8b990790c9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:30.933916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 862/1701] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:36:30.935251 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694028b-5f9d-47ed-884b-db8b990790c9 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:30.953498 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694028b-5f9d-47ed-884b-db8b990790c9 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:30.960370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeea5f1-ecea-42dc-bced-348c7065d72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:30.961486 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeea5f1-ecea-42dc-bced-348c7065d72b admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:30.962035 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeea5f1-ecea-42dc-bced-348c7065d72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:30.963140 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeea5f1-ecea-42dc-bced-348c7065d72b admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:30.964136 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeea5f1-ecea-42dc-bced-348c7065d72b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:30.967062 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeea5f1-ecea-42dc-bced-348c7065d72b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:30.983164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 863/1702] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:30.986192 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeea5f1-ecea-42dc-bced-348c7065d72b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:30.998786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeea5f1-ecea-42dc-bced-348c7065d72b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:31.007777 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dcd07-5abc-4d25-86b1-ea216003674e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:31.010027 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dcd07-5abc-4d25-86b1-ea216003674e admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:31.010027 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dcd07-5abc-4d25-86b1-ea216003674e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:31.010027 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dcd07-5abc-4d25-86b1-ea216003674e admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:31.010977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589dcd07-5abc-4d25-86b1-ea216003674e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:31.014343 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589dcd07-5abc-4d25-86b1-ea216003674e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:31.048968 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:31.069467 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-598039cf-59f2-46b5-88cc-9aafb3020da6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2895914d8bcc4960a646eac3df8d92f3'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:31.136054 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-598039cf-59f2-46b5-88cc-9aafb3020da6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2895914d8bcc4960a646eac3df8d92f3'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:31.139156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-598039cf-59f2-46b5-88cc-9aafb3020da6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2895914d8bcc4960a646eac3df8d92f3'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:31.143343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-598039cf-59f2-46b5-88cc-9aafb3020da6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2895914d8bcc4960a646eac3df8d92f3'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:31.161182 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 864/1703] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:30 2022] POST /identity/v3/projects => generated 403 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:31.161182 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dcd07-5abc-4d25-86b1-ea216003674e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:31.164973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-589dcd07-5abc-4d25-86b1-ea216003674e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:31.168078 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-598039cf-59f2-46b5-88cc-9aafb3020da6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2895914d8bcc4960a646eac3df8d92f3'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:31.248877 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 840/1704] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:36:30 2022] DELETE /identity/v3/users/2895914d8bcc4960a646eac3df8d92f3 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:31.249989 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598039cf-59f2-46b5-88cc-9aafb3020da6 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:31.253506 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-598039cf-59f2-46b5-88cc-9aafb3020da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:31.401434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dcd07-5abc-4d25-86b1-ea216003674e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:31.427409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598039cf-59f2-46b5-88cc-9aafb3020da6 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:31.440652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e9079-3661-41b0-8a18-5474007e807f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:31.441584 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e9079-3661-41b0-8a18-5474007e807f admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:31.441965 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e9079-3661-41b0-8a18-5474007e807f admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:31.442348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e9079-3661-41b0-8a18-5474007e807f admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:31.443226 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e9079-3661-41b0-8a18-5474007e807f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:31.446249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e9079-3661-41b0-8a18-5474007e807f admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:31.561633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c5137-d252-47b2-9379-f071fbed7295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:31.562235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c5137-d252-47b2-9379-f071fbed7295 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:31.562509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c5137-d252-47b2-9379-f071fbed7295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:31.562859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c5137-d252-47b2-9379-f071fbed7295 admin admin] PATH_INFO: `/v3/projects/37da538b9ed248e3aabb57e3e008794a/users/e8058e72bac2463694f49fc6db81ac30/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:31.625622 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c5137-d252-47b2-9379-f071fbed7295 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37da538b9ed248e3aabb57e3e008794a, user_id=e8058e72bac2463694f49fc6db81ac30, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:31.628629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c5137-d252-47b2-9379-f071fbed7295 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:31.654088 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 841/1705] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:31 2022] POST /identity/v3/users => generated 302 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:31.669692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e9079-3661-41b0-8a18-5474007e807f admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:31.692545 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e9079-3661-41b0-8a18-5474007e807f admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:31.701919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed8068-2ca7-4fca-9c16-a959c546d63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:31.702933 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed8068-2ca7-4fca-9c16-a959c546d63a admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:31.705680 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed8068-2ca7-4fca-9c16-a959c546d63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:31.708698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed8068-2ca7-4fca-9c16-a959c546d63a admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:31.708698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed8068-2ca7-4fca-9c16-a959c546d63a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:31.711020 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed8068-2ca7-4fca-9c16-a959c546d63a admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:31.737180 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 865/1706] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:31 2022] PUT /identity/v3/projects/37da538b9ed248e3aabb57e3e008794a/users/e8058e72bac2463694f49fc6db81ac30/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:31.741133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c5137-d252-47b2-9379-f071fbed7295 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:31.754682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 842/1707] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:31.755653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed8068-2ca7-4fca-9c16-a959c546d63a admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:31.782801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed8068-2ca7-4fca-9c16-a959c546d63a admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:31.912111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c5137-d252-47b2-9379-f071fbed7295 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:31.932550 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59bb0d-4761-47c8-a45a-7bc78b065697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:31.936900 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59bb0d-4761-47c8-a45a-7bc78b065697 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:31.938883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59bb0d-4761-47c8-a45a-7bc78b065697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:31.940967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59bb0d-4761-47c8-a45a-7bc78b065697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:31.941869 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59bb0d-4761-47c8-a45a-7bc78b065697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:31.954746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59bb0d-4761-47c8-a45a-7bc78b065697 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:31.980232 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 866/1708] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:31.982269 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59bb0d-4761-47c8-a45a-7bc78b065697 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:31.990064 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bb59bb0d-4761-47c8-a45a-7bc78b065697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:31.994750 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a20c85-e934-43fe-8d4e-a7e5fe7ea017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:31.996183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a20c85-e934-43fe-8d4e-a7e5fe7ea017 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:31.997985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a20c85-e934-43fe-8d4e-a7e5fe7ea017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:31.999087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a20c85-e934-43fe-8d4e-a7e5fe7ea017 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:32.000879 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a20c85-e934-43fe-8d4e-a7e5fe7ea017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:32.008884 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a20c85-e934-43fe-8d4e-a7e5fe7ea017 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:32.034683 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59bb0d-4761-47c8-a45a-7bc78b065697 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:32.039987 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 843/1709] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:32.040844 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a20c85-e934-43fe-8d4e-a7e5fe7ea017 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:32.048505 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cebe3a-4ae3-413e-b856-b89f35c9fb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:32.049179 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cebe3a-4ae3-413e-b856-b89f35c9fb9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:32.049518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cebe3a-4ae3-413e-b856-b89f35c9fb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:32.049782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cebe3a-4ae3-413e-b856-b89f35c9fb9d admin admin] PATH_INFO: `/v3/projects/9ef6a248373c47c0b7af49a1833c017a` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:32.052684 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cebe3a-4ae3-413e-b856-b89f35c9fb9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ef6a248373c47c0b7af49a1833c017a)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:32.054645 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cebe3a-4ae3-413e-b856-b89f35c9fb9d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:32.064629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a20c85-e934-43fe-8d4e-a7e5fe7ea017 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:32.087008 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ebe94-02e5-49ef-a573-ff8b338d8171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:32.087695 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ebe94-02e5-49ef-a573-ff8b338d8171 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:32.087975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ebe94-02e5-49ef-a573-ff8b338d8171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:32.088252 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ebe94-02e5-49ef-a573-ff8b338d8171 admin admin] PATH_INFO: `/v3/projects/37da538b9ed248e3aabb57e3e008794a/users/e8058e72bac2463694f49fc6db81ac30/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:32.099154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ebe94-02e5-49ef-a573-ff8b338d8171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37da538b9ed248e3aabb57e3e008794a, user_id=e8058e72bac2463694f49fc6db81ac30, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:32.107194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ebe94-02e5-49ef-a573-ff8b338d8171 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:32.126665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 844/1710] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:32 2022] PUT /identity/v3/projects/37da538b9ed248e3aabb57e3e008794a/users/e8058e72bac2463694f49fc6db81ac30/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:32.136804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ebe94-02e5-49ef-a573-ff8b338d8171 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:32.226451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-50cebe3a-4ae3-413e-b856-b89f35c9fb9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ef6a248373c47c0b7af49a1833c017a', 'request_id': 'req-50cebe3a-4ae3-413e-b856-b89f35c9fb9d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:32.248669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-50cebe3a-4ae3-413e-b856-b89f35c9fb9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ef6a248373c47c0b7af49a1833c017a', 'request_id': 'req-50cebe3a-4ae3-413e-b856-b89f35c9fb9d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:32.269241 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-50cebe3a-4ae3-413e-b856-b89f35c9fb9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ef6a248373c47c0b7af49a1833c017a', 'request_id': 'req-50cebe3a-4ae3-413e-b856-b89f35c9fb9d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:32.280149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ebe94-02e5-49ef-a573-ff8b338d8171 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:32.376513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-50cebe3a-4ae3-413e-b856-b89f35c9fb9d admin admin] The token cache is being invalidate because project 9ef6a248373c47c0b7af49a1833c017a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:36:32.384144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-50cebe3a-4ae3-413e-b856-b89f35c9fb9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:32.384144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-50cebe3a-4ae3-413e-b856-b89f35c9fb9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:32.384144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 867/1711] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:36:31 2022] DELETE /identity/v3/projects/9ef6a248373c47c0b7af49a1833c017a => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:32.384695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c7bdc-7c5f-4bcf-baa4-ecfd218566a4 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:32.385939 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c7bdc-7c5f-4bcf-baa4-ecfd218566a4 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:32.386304 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c7bdc-7c5f-4bcf-baa4-ecfd218566a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:32.394542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c8dac-2909-4ecd-99bb-353ed29f4f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:32.395535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c8dac-2909-4ecd-99bb-353ed29f4f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:32.395806 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c8dac-2909-4ecd-99bb-353ed29f4f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:32.396252 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c8dac-2909-4ecd-99bb-353ed29f4f11 admin admin] PATH_INFO: `/v3/projects/7eaac096ac0642eb8b1d456b029627ab/users/676dd2bec5db41bf8982062b89211a48/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:32.411201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:32.422834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c8dac-2909-4ecd-99bb-353ed29f4f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eaac096ac0642eb8b1d456b029627ab, user_id=676dd2bec5db41bf8982062b89211a48, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:32.426064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c8dac-2909-4ecd-99bb-353ed29f4f11 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:32.475975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8a7c7bdc-7c5f-4bcf-baa4-ecfd218566a4 None None] MFA Rules not processed for user `e8058e72bac2463694f49fc6db81ac30`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:32.475975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8a7c7bdc-7c5f-4bcf-baa4-ecfd218566a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:32.546419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 845/1712] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:32 2022] PUT /identity/v3/projects/7eaac096ac0642eb8b1d456b029627ab/users/676dd2bec5db41bf8982062b89211a48/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:32.547477 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c8dac-2909-4ecd-99bb-353ed29f4f11 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:32.557473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-2d6c8dac-2909-4ecd-99bb-353ed29f4f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:32.635948 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 868/1713] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:32 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:32.638414 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb054eb-37fa-4488-987b-f028afe57777 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:32.639105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb054eb-37fa-4488-987b-f028afe57777 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:32.647738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb054eb-37fa-4488-987b-f028afe57777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:32.690764 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4bb054eb-37fa-4488-987b-f028afe57777 None None] MFA Rules not processed for user `676dd2bec5db41bf8982062b89211a48`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:32.693280 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4bb054eb-37fa-4488-987b-f028afe57777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:32.739650 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c8dac-2909-4ecd-99bb-353ed29f4f11 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:32.753146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b68ac-8b23-471f-ac67-8fe1362de445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:32.753824 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b68ac-8b23-471f-ac67-8fe1362de445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:32.754093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b68ac-8b23-471f-ac67-8fe1362de445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:32.754369 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b68ac-8b23-471f-ac67-8fe1362de445 admin admin] PATH_INFO: `/v3/users/c6c7573c169b4415ac4cd2153e89d3b1` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:32.757810 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b68ac-8b23-471f-ac67-8fe1362de445 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6c7573c169b4415ac4cd2153e89d3b1)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:32.763783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b68ac-8b23-471f-ac67-8fe1362de445 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:32.853462 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 869/1714] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:32 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:32.854578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb054eb-37fa-4488-987b-f028afe57777 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:32.857720 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4bb054eb-37fa-4488-987b-f028afe57777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:32.932609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb054eb-37fa-4488-987b-f028afe57777 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:32.962006 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1d41f-a38f-4854-8507-6384e2b9f97e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:32.962840 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1d41f-a38f-4854-8507-6384e2b9f97e service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:32.963266 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1d41f-a38f-4854-8507-6384e2b9f97e service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:32.963931 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1d41f-a38f-4854-8507-6384e2b9f97e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:33.059856 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-923b68ac-8b23-471f-ac67-8fe1362de445 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6c7573c169b4415ac4cd2153e89d3b1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:33.116616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1d41f-a38f-4854-8507-6384e2b9f97e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:33.120698 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1d41f-a38f-4854-8507-6384e2b9f97e service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:33.153695 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-923b68ac-8b23-471f-ac67-8fe1362de445 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6c7573c169b4415ac4cd2153e89d3b1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:33.154350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-923b68ac-8b23-471f-ac67-8fe1362de445 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6c7573c169b4415ac4cd2153e89d3b1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:33.154972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-923b68ac-8b23-471f-ac67-8fe1362de445 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6c7573c169b4415ac4cd2153e89d3b1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:33.168571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-923b68ac-8b23-471f-ac67-8fe1362de445 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6c7573c169b4415ac4cd2153e89d3b1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:33.172332 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 870/1715] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:32 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:33.174425 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1d41f-a38f-4854-8507-6384e2b9f97e service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:33.178049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3fa1d41f-a38f-4854-8507-6384e2b9f97e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:33.204237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1d41f-a38f-4854-8507-6384e2b9f97e service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:33.205783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 846/1716] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:36:32 2022] DELETE /identity/v3/users/c6c7573c169b4415ac4cd2153e89d3b1 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:33.427260 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007c102-65c4-425e-be62-9aed83e4c315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:33.427953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007c102-65c4-425e-be62-9aed83e4c315 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:33.428239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007c102-65c4-425e-be62-9aed83e4c315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:33.428512 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007c102-65c4-425e-be62-9aed83e4c315 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:33.429277 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007c102-65c4-425e-be62-9aed83e4c315 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:33.431608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007c102-65c4-425e-be62-9aed83e4c315 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:33.566679 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 871/1717] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:33 2022] POST /identity/v3/projects => generated 405 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:33.571271 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b68ac-8b23-471f-ac67-8fe1362de445 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:33.573166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-923b68ac-8b23-471f-ac67-8fe1362de445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:33.602287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b68ac-8b23-471f-ac67-8fe1362de445 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:33.627568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e9c74-4c12-4ae5-a81f-ab583e183554 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:33.627884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e9c74-4c12-4ae5-a81f-ab583e183554 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:33.628176 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e9c74-4c12-4ae5-a81f-ab583e183554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:33.718152 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c7a5d-b2d7-4918-b1c3-a70e2a8c24be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:33.719083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c7a5d-b2d7-4918-b1c3-a70e2a8c24be admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:33.719637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c7a5d-b2d7-4918-b1c3-a70e2a8c24be admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:33.720064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c7a5d-b2d7-4918-b1c3-a70e2a8c24be admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:33.720881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c7a5d-b2d7-4918-b1c3-a70e2a8c24be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:33.725345 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8d5e9c74-4c12-4ae5-a81f-ab583e183554 None None] MFA Rules not processed for user `6712334ebb77462daa89be58d4480846`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:33.730065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8d5e9c74-4c12-4ae5-a81f-ab583e183554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:33.755141 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c7a5d-b2d7-4918-b1c3-a70e2a8c24be admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:33.785251 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:33.884367 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 847/1718] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:33 2022] POST /identity/v3/users => generated 309 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:33.885592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c7a5d-b2d7-4918-b1c3-a70e2a8c24be admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:33.915292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c7a5d-b2d7-4918-b1c3-a70e2a8c24be admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:33.952562 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b41c7-353a-457a-9dbe-a56bd98738c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:33.953524 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b41c7-353a-457a-9dbe-a56bd98738c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:33.954880 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b41c7-353a-457a-9dbe-a56bd98738c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:33.955312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b41c7-353a-457a-9dbe-a56bd98738c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:33.956864 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b41c7-353a-457a-9dbe-a56bd98738c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:33.962369 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b41c7-353a-457a-9dbe-a56bd98738c4 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:33.981454 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 872/1719] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:33.996701 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 848/1720] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:33.998231 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e9c74-4c12-4ae5-a81f-ab583e183554 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:33.999336 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b41c7-353a-457a-9dbe-a56bd98738c4 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:33.999716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8d5e9c74-4c12-4ae5-a81f-ab583e183554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:34.032001 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b41c7-353a-457a-9dbe-a56bd98738c4 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:34.052251 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56d992-e1ec-4f9c-b9a9-cca340509f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:34.059579 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56d992-e1ec-4f9c-b9a9-cca340509f36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:34.059579 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56d992-e1ec-4f9c-b9a9-cca340509f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:34.059579 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56d992-e1ec-4f9c-b9a9-cca340509f36 admin admin] PATH_INFO: `/v3/projects/506c707baa5e459b888946eddaa62583/users/72f1f84cc2b44a2ea64303b044f798bf/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:34.141471 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56d992-e1ec-4f9c-b9a9-cca340509f36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=506c707baa5e459b888946eddaa62583, user_id=72f1f84cc2b44a2ea64303b044f798bf, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:34.149780 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56d992-e1ec-4f9c-b9a9-cca340509f36 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:34.221919 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e9c74-4c12-4ae5-a81f-ab583e183554 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:34.231227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:34.240848 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c8cdf-9245-4b3d-b449-2e560c19d0d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:34.241753 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c8cdf-9245-4b3d-b449-2e560c19d0d0 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:34.242239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c8cdf-9245-4b3d-b449-2e560c19d0d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:34.242659 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c8cdf-9245-4b3d-b449-2e560c19d0d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:34.259439 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c8cdf-9245-4b3d-b449-2e560c19d0d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:34.261816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c8cdf-9245-4b3d-b449-2e560c19d0d0 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:34.275224 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 849/1721] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:33 2022] PUT /identity/v3/projects/506c707baa5e459b888946eddaa62583/users/72f1f84cc2b44a2ea64303b044f798bf/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:34.277149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56d992-e1ec-4f9c-b9a9-cca340509f36 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:34.342906 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 873/1722] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:36:34.524614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56d992-e1ec-4f9c-b9a9-cca340509f36 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:34.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd718158-1131-418d-b600-e34c552a19aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:34.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd718158-1131-418d-b600-e34c552a19aa admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:34.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd718158-1131-418d-b600-e34c552a19aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:34.540788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd718158-1131-418d-b600-e34c552a19aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:34.546715 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd718158-1131-418d-b600-e34c552a19aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:34.548721 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd718158-1131-418d-b600-e34c552a19aa admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:34.578065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c8cdf-9245-4b3d-b449-2e560c19d0d0 service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:34.582440 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 850/1723] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:34.609041 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c8cdf-9245-4b3d-b449-2e560c19d0d0 service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:34.639140 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33ff2b-4bfd-4e72-8aa7-ed4df86a53d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:34.644662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33ff2b-4bfd-4e72-8aa7-ed4df86a53d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:34.644662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33ff2b-4bfd-4e72-8aa7-ed4df86a53d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:34.644662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33ff2b-4bfd-4e72-8aa7-ed4df86a53d2 admin admin] PATH_INFO: `/v3/projects/506c707baa5e459b888946eddaa62583/users/72f1f84cc2b44a2ea64303b044f798bf/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:34.711023 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33ff2b-4bfd-4e72-8aa7-ed4df86a53d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=506c707baa5e459b888946eddaa62583, user_id=72f1f84cc2b44a2ea64303b044f798bf, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:34.711023 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33ff2b-4bfd-4e72-8aa7-ed4df86a53d2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:34.717348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 874/1724] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:34 2022] PUT /identity/v3/projects/506c707baa5e459b888946eddaa62583/users/72f1f84cc2b44a2ea64303b044f798bf/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:34.737665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0592fe3-0831-419f-8a34-401839ffacd9 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:34.737957 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0592fe3-0831-419f-8a34-401839ffacd9 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:34.739090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0592fe3-0831-419f-8a34-401839ffacd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:34.860752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-b0592fe3-0831-419f-8a34-401839ffacd9 None None] MFA Rules not processed for user `72f1f84cc2b44a2ea64303b044f798bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:34.864546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b0592fe3-0831-419f-8a34-401839ffacd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:35.133207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 851/1725] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:34 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:35.207876 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33ff2b-4bfd-4e72-8aa7-ed4df86a53d2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:35.209744 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2c33ff2b-4bfd-4e72-8aa7-ed4df86a53d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:35.259344 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0592fe3-0831-419f-8a34-401839ffacd9 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:35.260934 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b0592fe3-0831-419f-8a34-401839ffacd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:35.384561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33ff2b-4bfd-4e72-8aa7-ed4df86a53d2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:35.395588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b20749-b6ee-4e23-b36f-34bd7b8bf710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:35.397608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b20749-b6ee-4e23-b36f-34bd7b8bf710 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:35.397908 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b20749-b6ee-4e23-b36f-34bd7b8bf710 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:35.398188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b20749-b6ee-4e23-b36f-34bd7b8bf710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:35.432835 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b20749-b6ee-4e23-b36f-34bd7b8bf710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:35.435096 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b20749-b6ee-4e23-b36f-34bd7b8bf710 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:35.458021 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0592fe3-0831-419f-8a34-401839ffacd9 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:35.458509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 875/1726] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:35 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:35.473747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee1cab-da90-4d14-ab3e-e8b72fe21c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:35.473747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee1cab-da90-4d14-ab3e-e8b72fe21c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:35.473747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee1cab-da90-4d14-ab3e-e8b72fe21c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:35.473747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee1cab-da90-4d14-ab3e-e8b72fe21c4e admin admin] PATH_INFO: `/v3/projects/e5e8ea57164d47868456cc09e2e4e415` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:35.474343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee1cab-da90-4d14-ab3e-e8b72fe21c4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5e8ea57164d47868456cc09e2e4e415)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:35.476399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee1cab-da90-4d14-ab3e-e8b72fe21c4e admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:35.522199 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:35.712606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-69ee1cab-da90-4d14-ab3e-e8b72fe21c4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5e8ea57164d47868456cc09e2e4e415', 'request_id': 'req-69ee1cab-da90-4d14-ab3e-e8b72fe21c4e'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:35.734202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-69ee1cab-da90-4d14-ab3e-e8b72fe21c4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5e8ea57164d47868456cc09e2e4e415', 'request_id': 'req-69ee1cab-da90-4d14-ab3e-e8b72fe21c4e'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:35.753529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:35.756560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-69ee1cab-da90-4d14-ab3e-e8b72fe21c4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5e8ea57164d47868456cc09e2e4e415', 'request_id': 'req-69ee1cab-da90-4d14-ab3e-e8b72fe21c4e'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:35.828088 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b20749-b6ee-4e23-b36f-34bd7b8bf710 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:35.830580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-26b20749-b6ee-4e23-b36f-34bd7b8bf710 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:35.846757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-69ee1cab-da90-4d14-ab3e-e8b72fe21c4e admin admin] The token cache is being invalidate because project e5e8ea57164d47868456cc09e2e4e415 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:36:35.848877 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b20749-b6ee-4e23-b36f-34bd7b8bf710 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:35.849107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-69ee1cab-da90-4d14-ab3e-e8b72fe21c4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:35.849107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-69ee1cab-da90-4d14-ab3e-e8b72fe21c4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:35.852259 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 852/1727] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:36:35 2022] DELETE /identity/v3/projects/e5e8ea57164d47868456cc09e2e4e415 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:35.858423 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff6afc-1e00-4db9-8fbf-13d4eae9933a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:35.859010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff6afc-1e00-4db9-8fbf-13d4eae9933a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:35.859478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff6afc-1e00-4db9-8fbf-13d4eae9933a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:35.884789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-8aff6afc-1e00-4db9-8fbf-13d4eae9933a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:35.898057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-8aff6afc-1e00-4db9-8fbf-13d4eae9933a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:35.961557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295204ac-43b7-46fe-974c-8d0f885eb78c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:35.974927 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295204ac-43b7-46fe-974c-8d0f885eb78c service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:35.976044 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295204ac-43b7-46fe-974c-8d0f885eb78c service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:35.978562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295204ac-43b7-46fe-974c-8d0f885eb78c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:35.983189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-295204ac-43b7-46fe-974c-8d0f885eb78c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:36.028368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 853/1728] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:36:36.034801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff6afc-1e00-4db9-8fbf-13d4eae9933a None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:36.041986 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-8aff6afc-1e00-4db9-8fbf-13d4eae9933a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:36.075996 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff6afc-1e00-4db9-8fbf-13d4eae9933a None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:36.089246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21850652-726f-40dc-83c4-4bcd4713c56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:36.090014 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21850652-726f-40dc-83c4-4bcd4713c56a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:36.090358 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21850652-726f-40dc-83c4-4bcd4713c56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:36.090909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21850652-726f-40dc-83c4-4bcd4713c56a admin admin] PATH_INFO: `/v3/users/676dd2bec5db41bf8982062b89211a48` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:36.092172 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21850652-726f-40dc-83c4-4bcd4713c56a admin admin] RBAC: Authorizing `identity:delete_user(user_id=676dd2bec5db41bf8982062b89211a48)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:36.094324 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21850652-726f-40dc-83c4-4bcd4713c56a admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:36.133524 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295204ac-43b7-46fe-974c-8d0f885eb78c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:36.135545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295204ac-43b7-46fe-974c-8d0f885eb78c service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:36.150603 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 876/1729] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:36.164339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bef51-9ce6-4af9-af95-2cd71aa59cf4 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:36.165073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bef51-9ce6-4af9-af95-2cd71aa59cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:36.166361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bef51-9ce6-4af9-af95-2cd71aa59cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:36.233160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-d91bef51-9ce6-4af9-af95-2cd71aa59cf4 None None] MFA Rules not processed for user `5fecaefd115842fe80009d70921c3806`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:36.235964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d91bef51-9ce6-4af9-af95-2cd71aa59cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:36.364479 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-21850652-726f-40dc-83c4-4bcd4713c56a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '676dd2bec5db41bf8982062b89211a48'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:36.371509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:36.421938 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 877/1730] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:36.423185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a87adf-5200-4416-8f83-abf901c65811 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:36.423474 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a87adf-5200-4416-8f83-abf901c65811 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:36.425092 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a87adf-5200-4416-8f83-abf901c65811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:36.472915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-21850652-726f-40dc-83c4-4bcd4713c56a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '676dd2bec5db41bf8982062b89211a48'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:36.473625 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-21850652-726f-40dc-83c4-4bcd4713c56a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '676dd2bec5db41bf8982062b89211a48'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:36.474253 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-21850652-726f-40dc-83c4-4bcd4713c56a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '676dd2bec5db41bf8982062b89211a48'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:36.481404 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f4a87adf-5200-4416-8f83-abf901c65811 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:36.485567 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f4a87adf-5200-4416-8f83-abf901c65811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:36.486125 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-21850652-726f-40dc-83c4-4bcd4713c56a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '676dd2bec5db41bf8982062b89211a48'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:36.500280 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 878/1731] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:36.501527 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf0a7e-e39d-4002-9628-bfd6ab320031 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:36.501810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf0a7e-e39d-4002-9628-bfd6ab320031 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:36.502081 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf0a7e-e39d-4002-9628-bfd6ab320031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:36.511501 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 854/1732] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:36:36 2022] DELETE /identity/v3/users/676dd2bec5db41bf8982062b89211a48 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:36.515911 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21850652-726f-40dc-83c4-4bcd4713c56a admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:36.532861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-02cf0a7e-e39d-4002-9628-bfd6ab320031 None None] MFA Rules not processed for user `5fecaefd115842fe80009d70921c3806`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:36.534367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-02cf0a7e-e39d-4002-9628-bfd6ab320031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:36.661976 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 879/1733] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:36.663197 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c84fd9-8703-4040-bea5-767e1d6381e5 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:36.663678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c84fd9-8703-4040-bea5-767e1d6381e5 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:36.664129 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c84fd9-8703-4040-bea5-767e1d6381e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:36.685936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21850652-726f-40dc-83c4-4bcd4713c56a admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:36.688033 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-25c84fd9-8703-4040-bea5-767e1d6381e5 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:36.689803 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-25c84fd9-8703-4040-bea5-767e1d6381e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:36.691650 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137529a-34ab-4921-9080-d7ac5f7f39be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:36.692499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137529a-34ab-4921-9080-d7ac5f7f39be admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:36.693035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137529a-34ab-4921-9080-d7ac5f7f39be admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:36.706099 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137529a-34ab-4921-9080-d7ac5f7f39be admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:36.706099 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137529a-34ab-4921-9080-d7ac5f7f39be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:36:36.706099 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137529a-34ab-4921-9080-d7ac5f7f39be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:36.706099 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137529a-34ab-4921-9080-d7ac5f7f39be admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:36.706992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 880/1734] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:36.706992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c84fd9-8703-4040-bea5-767e1d6381e5 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:36.706992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-25c84fd9-8703-4040-bea5-767e1d6381e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:36.736073 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 855/1735] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:36:36.738851 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137529a-34ab-4921-9080-d7ac5f7f39be admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:36.742460 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d137529a-34ab-4921-9080-d7ac5f7f39be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:36.932259 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137529a-34ab-4921-9080-d7ac5f7f39be admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:36.941470 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2220bd-71e1-43f3-8605-05e38250a490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:36.943447 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2220bd-71e1-43f3-8605-05e38250a490 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:36.943731 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2220bd-71e1-43f3-8605-05e38250a490 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:36.943991 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2220bd-71e1-43f3-8605-05e38250a490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:36.960273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c84fd9-8703-4040-bea5-767e1d6381e5 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:36.972824 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2220bd-71e1-43f3-8605-05e38250a490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:36.975203 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2220bd-71e1-43f3-8605-05e38250a490 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:37.001270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29ed7a-ce5a-4ee9-b65a-4eb19a185948 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:37.001270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29ed7a-ce5a-4ee9-b65a-4eb19a185948 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:37.001270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29ed7a-ce5a-4ee9-b65a-4eb19a185948 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:37.002796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29ed7a-ce5a-4ee9-b65a-4eb19a185948 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:37.004199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 856/1736] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:37.006473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-eb29ed7a-ce5a-4ee9-b65a-4eb19a185948 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:37.008775 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2220bd-71e1-43f3-8605-05e38250a490 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:37.019977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29ed7a-ce5a-4ee9-b65a-4eb19a185948 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:37.020878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2220bd-71e1-43f3-8605-05e38250a490 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:37.021964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29ed7a-ce5a-4ee9-b65a-4eb19a185948 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:37.026399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253248e0-a333-4d0c-a7e9-6eb14b0496f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:37.027783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253248e0-a333-4d0c-a7e9-6eb14b0496f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:37.028387 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253248e0-a333-4d0c-a7e9-6eb14b0496f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:37.028807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253248e0-a333-4d0c-a7e9-6eb14b0496f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:37.029712 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253248e0-a333-4d0c-a7e9-6eb14b0496f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:37.033487 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253248e0-a333-4d0c-a7e9-6eb14b0496f0 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:37.034407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 881/1737] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:37.121125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81e198-46d7-475d-90b9-f0af8bc028e3 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:37.121542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81e198-46d7-475d-90b9-f0af8bc028e3 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:37.126202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81e198-46d7-475d-90b9-f0af8bc028e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:37.150078 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 857/1738] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:37 2022] POST /identity/v3/projects => generated 365 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:37.154989 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253248e0-a333-4d0c-a7e9-6eb14b0496f0 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:37.178749 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1e81e198-46d7-475d-90b9-f0af8bc028e3 None None] MFA Rules not processed for user `8a059ba74d744a12a4694513f8ca5789`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:37.186169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1e81e198-46d7-475d-90b9-f0af8bc028e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:37.317982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 882/1739] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:37.379612 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253248e0-a333-4d0c-a7e9-6eb14b0496f0 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:37.384346 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a97dd-01aa-417c-9c20-9c8af9f84b7c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:37.384631 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a97dd-01aa-417c-9c20-9c8af9f84b7c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:37.384904 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a97dd-01aa-417c-9c20-9c8af9f84b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:37.388633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf488b-595b-4009-a431-39411a4b930f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:37.390402 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf488b-595b-4009-a431-39411a4b930f admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:37.390941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf488b-595b-4009-a431-39411a4b930f admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:37.391415 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf488b-595b-4009-a431-39411a4b930f admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:37.392724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf488b-595b-4009-a431-39411a4b930f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:37.395831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf488b-595b-4009-a431-39411a4b930f admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:37.445120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a62a97dd-01aa-417c-9c20-9c8af9f84b7c None None] MFA Rules not processed for user `d4d9bea213fa4401992d8e7894726e05`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:37.446507 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a62a97dd-01aa-417c-9c20-9c8af9f84b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:37.519073 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 858/1740] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:37 2022] POST /identity/v3/users => generated 283 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:37.520485 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf488b-595b-4009-a431-39411a4b930f admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:37.532361 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf488b-595b-4009-a431-39411a4b930f admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:37.541907 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7b78e-b246-4983-afd5-1d6c86f6e2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:37.542522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7b78e-b246-4983-afd5-1d6c86f6e2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:37.543423 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7b78e-b246-4983-afd5-1d6c86f6e2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:37.543980 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7b78e-b246-4983-afd5-1d6c86f6e2c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:37.544727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b7b78e-b246-4983-afd5-1d6c86f6e2c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:37.546687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b7b78e-b246-4983-afd5-1d6c86f6e2c4 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:37.579748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 859/1741] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:37.583896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7b78e-b246-4983-afd5-1d6c86f6e2c4 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:37.599356 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7b78e-b246-4983-afd5-1d6c86f6e2c4 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:37.615413 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9ac65-0d73-4a2d-9cf8-03e1ce810cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:37.617095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9ac65-0d73-4a2d-9cf8-03e1ce810cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:37.619192 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9ac65-0d73-4a2d-9cf8-03e1ce810cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:37.619655 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9ac65-0d73-4a2d-9cf8-03e1ce810cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:37.621380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9ac65-0d73-4a2d-9cf8-03e1ce810cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:37.623261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9ac65-0d73-4a2d-9cf8-03e1ce810cc3 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:37.648416 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 860/1742] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:37.651246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9ac65-0d73-4a2d-9cf8-03e1ce810cc3 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:37.654521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 883/1743] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:37.697528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9ac65-0d73-4a2d-9cf8-03e1ce810cc3 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:37.703805 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07579df0-06b8-4c62-b2a0-967345745121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:37.704873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07579df0-06b8-4c62-b2a0-967345745121 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:37.705662 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07579df0-06b8-4c62-b2a0-967345745121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:37.706297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07579df0-06b8-4c62-b2a0-967345745121 admin admin] PATH_INFO: `/v3/projects/5bffeada6f994879a6475b975463c0f9/users/02e8375c12cc4c7f8c999eb9c5823c21/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:37.727243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07579df0-06b8-4c62-b2a0-967345745121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bffeada6f994879a6475b975463c0f9, user_id=02e8375c12cc4c7f8c999eb9c5823c21, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:37.731751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07579df0-06b8-4c62-b2a0-967345745121 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:37.739943 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:37.880100 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5ab20-395d-4e7d-b3d1-42632e5db0ba None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:37.880394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5ab20-395d-4e7d-b3d1-42632e5db0ba None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:37.880671 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5ab20-395d-4e7d-b3d1-42632e5db0ba None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:37.882328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 884/1744] 10.222.0.42 () {54 vars in 912 bytes} [Mon Sep 19 10:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:36:37.896140 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aeace5-1774-40b3-a7ab-54c2c8044940 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:37.896140 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aeace5-1774-40b3-a7ab-54c2c8044940 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:37.900961 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aeace5-1774-40b3-a7ab-54c2c8044940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:37.915695 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 861/1745] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:37 2022] PUT /identity/v3/projects/5bffeada6f994879a6475b975463c0f9/users/02e8375c12cc4c7f8c999eb9c5823c21/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:37.926531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b6254-e62e-48a3-aed6-aae4832509a4 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:37.926904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b6254-e62e-48a3-aed6-aae4832509a4 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:37.927253 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b6254-e62e-48a3-aed6-aae4832509a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:37.975674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-0b7b6254-e62e-48a3-aed6-aae4832509a4 None None] MFA Rules not processed for user `02e8375c12cc4c7f8c999eb9c5823c21`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:37.976590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-16aeace5-1774-40b3-a7ab-54c2c8044940 None None] MFA Rules not processed for user `a282e74d177544bbbfba0fdb5e52df60`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:37.977621 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-0b7b6254-e62e-48a3-aed6-aae4832509a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:37.978159 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-16aeace5-1774-40b3-a7ab-54c2c8044940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:38.096256 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 885/1746] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 19 10:36:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:36:38.112466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aeace5-1774-40b3-a7ab-54c2c8044940 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:38.114409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-16aeace5-1774-40b3-a7ab-54c2c8044940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:38.127833 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aeace5-1774-40b3-a7ab-54c2c8044940 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:38.132354 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87868431-6a4d-48bb-8a7a-87408df79a1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:38.133353 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 862/1747] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:38.136821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87868431-6a4d-48bb-8a7a-87408df79a1a service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:38.137170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87868431-6a4d-48bb-8a7a-87408df79a1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:38.143907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87868431-6a4d-48bb-8a7a-87408df79a1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:38.144064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b6254-e62e-48a3-aed6-aae4832509a4 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:38.147531 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87868431-6a4d-48bb-8a7a-87408df79a1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:38.150181 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87868431-6a4d-48bb-8a7a-87408df79a1a service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:38.171668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 886/1748] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:36:38.291058 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b6254-e62e-48a3-aed6-aae4832509a4 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:38.291058 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24b523-ba65-4de6-8cfb-3689d0d663de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:38.291058 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24b523-ba65-4de6-8cfb-3689d0d663de admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:38.291058 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24b523-ba65-4de6-8cfb-3689d0d663de admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:38.292571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24b523-ba65-4de6-8cfb-3689d0d663de admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:38.292571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24b523-ba65-4de6-8cfb-3689d0d663de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:38.292571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24b523-ba65-4de6-8cfb-3689d0d663de admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:38.543429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87868431-6a4d-48bb-8a7a-87408df79a1a service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:38.550972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 863/1749] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:38 2022] POST /identity/v3/projects => generated 365 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:36:38.594297 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87868431-6a4d-48bb-8a7a-87408df79a1a service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:38.805587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7a724-7f03-4533-874d-3bd530e94f85 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:38.806114 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7a724-7f03-4533-874d-3bd530e94f85 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:38.806554 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7a724-7f03-4533-874d-3bd530e94f85 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:38.809857 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 864/1750] 10.222.0.42 () {54 vars in 912 bytes} [Mon Sep 19 10:36:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:36:38.815622 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da473d-49e5-49bf-8022-b24c5327c24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:38.817344 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd855cff-c7ff-4153-91a8-56e06b2500d7 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:38.825222 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da473d-49e5-49bf-8022-b24c5327c24f admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:38.825222 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da473d-49e5-49bf-8022-b24c5327c24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:38.825531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd855cff-c7ff-4153-91a8-56e06b2500d7 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:38.825531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd855cff-c7ff-4153-91a8-56e06b2500d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:38.830749 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da473d-49e5-49bf-8022-b24c5327c24f admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:38.831101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da473d-49e5-49bf-8022-b24c5327c24f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:38.835998 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da473d-49e5-49bf-8022-b24c5327c24f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:38.859678 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-cd855cff-c7ff-4153-91a8-56e06b2500d7 None None] MFA Rules not processed for user `a282e74d177544bbbfba0fdb5e52df60`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:38.859678 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-cd855cff-c7ff-4153-91a8-56e06b2500d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:38.907584 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:38.951037 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 887/1751] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:38 2022] POST /identity/v3/users => generated 289 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:38.952186 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da473d-49e5-49bf-8022-b24c5327c24f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:39.000871 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 865/1752] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 19 10:36:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:36:39.002029 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd855cff-c7ff-4153-91a8-56e06b2500d7 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:39.003575 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-cd855cff-c7ff-4153-91a8-56e06b2500d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:39.020880 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd855cff-c7ff-4153-91a8-56e06b2500d7 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:39.026482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7e840-082e-42d5-94ea-12b8ca9918f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:39.027326 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7e840-082e-42d5-94ea-12b8ca9918f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:39.027905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7e840-082e-42d5-94ea-12b8ca9918f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:39.027905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7e840-082e-42d5-94ea-12b8ca9918f5 admin admin] PATH_INFO: `/v3/projects/7eaac096ac0642eb8b1d456b029627ab` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:39.029503 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c7e840-082e-42d5-94ea-12b8ca9918f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eaac096ac0642eb8b1d456b029627ab)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:39.031634 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c7e840-082e-42d5-94ea-12b8ca9918f5 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:39.117438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da473d-49e5-49bf-8022-b24c5327c24f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:39.131726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6c250-c85a-417b-9524-59b012ecc5dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:39.132394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6c250-c85a-417b-9524-59b012ecc5dd service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:39.133200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6c250-c85a-417b-9524-59b012ecc5dd service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:39.133200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6c250-c85a-417b-9524-59b012ecc5dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:39.217296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-e1c7e840-082e-42d5-94ea-12b8ca9918f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eaac096ac0642eb8b1d456b029627ab', 'request_id': 'req-e1c7e840-082e-42d5-94ea-12b8ca9918f5'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:39.227756 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b6c250-c85a-417b-9524-59b012ecc5dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:39.230723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b6c250-c85a-417b-9524-59b012ecc5dd service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:39.231223 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-e1c7e840-082e-42d5-94ea-12b8ca9918f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eaac096ac0642eb8b1d456b029627ab', 'request_id': 'req-e1c7e840-082e-42d5-94ea-12b8ca9918f5'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:39.243979 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 888/1753] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:39.245512 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6c250-c85a-417b-9524-59b012ecc5dd service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:39.258219 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6c250-c85a-417b-9524-59b012ecc5dd service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:39.263071 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-e1c7e840-082e-42d5-94ea-12b8ca9918f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eaac096ac0642eb8b1d456b029627ab', 'request_id': 'req-e1c7e840-082e-42d5-94ea-12b8ca9918f5'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:39.372059 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd2d2e-a6d3-4596-a1e5-00ce7e64ae17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:39.372985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd2d2e-a6d3-4596-a1e5-00ce7e64ae17 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:39.373302 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd2d2e-a6d3-4596-a1e5-00ce7e64ae17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:39.373583 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd2d2e-a6d3-4596-a1e5-00ce7e64ae17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:39.374246 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd2d2e-a6d3-4596-a1e5-00ce7e64ae17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:39.375911 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd2d2e-a6d3-4596-a1e5-00ce7e64ae17 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:39.377168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:39.383385 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-e1c7e840-082e-42d5-94ea-12b8ca9918f5 admin admin] The token cache is being invalidate because project 7eaac096ac0642eb8b1d456b029627ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:36:39.383923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-e1c7e840-082e-42d5-94ea-12b8ca9918f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:39.384875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-e1c7e840-082e-42d5-94ea-12b8ca9918f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:39.387496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 866/1754] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:36:39 2022] DELETE /identity/v3/projects/7eaac096ac0642eb8b1d456b029627ab => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:39.388462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7e840-082e-42d5-94ea-12b8ca9918f5 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:39.390109 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e1c7e840-082e-42d5-94ea-12b8ca9918f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:39.394944 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 889/1755] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:39.400079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd2d2e-a6d3-4596-a1e5-00ce7e64ae17 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:39.408473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f5dd2d2e-a6d3-4596-a1e5-00ce7e64ae17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:39.526338 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7e840-082e-42d5-94ea-12b8ca9918f5 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:39.554199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b27de-8ae0-4deb-a790-51054793d4b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0ef47a60f54ca1853fd267e308c58b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:39.555612 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b27de-8ae0-4deb-a790-51054793d4b1 service placement] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:39.556083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b27de-8ae0-4deb-a790-51054793d4b1 service placement] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:39.557872 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b27de-8ae0-4deb-a790-51054793d4b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:39.564339 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4b8b27de-8ae0-4deb-a790-51054793d4b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:39.579511 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b27de-8ae0-4deb-a790-51054793d4b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:39.589254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b27de-8ae0-4deb-a790-51054793d4b1 service placement] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:39.598046 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 867/1756] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:39.603966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b27de-8ae0-4deb-a790-51054793d4b1 service placement] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:39.603966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4b8b27de-8ae0-4deb-a790-51054793d4b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:39.638633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b27de-8ae0-4deb-a790-51054793d4b1 service placement] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:39.652726 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6c610-5f8f-4fa8-bf66-2bc6d55a7952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:39.653422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6c610-5f8f-4fa8-bf66-2bc6d55a7952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:39.653750 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6c610-5f8f-4fa8-bf66-2bc6d55a7952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:39.654033 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6c610-5f8f-4fa8-bf66-2bc6d55a7952 admin admin] PATH_INFO: `/v3/users/72f1f84cc2b44a2ea64303b044f798bf` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:39.655895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6c610-5f8f-4fa8-bf66-2bc6d55a7952 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72f1f84cc2b44a2ea64303b044f798bf)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:39.658026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6c610-5f8f-4fa8-bf66-2bc6d55a7952 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:39.705139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd2d2e-a6d3-4596-a1e5-00ce7e64ae17 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:39.718492 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a29f1-6e9a-457b-b4e4-03d5b5ee80cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:39.723470 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a29f1-6e9a-457b-b4e4-03d5b5ee80cd service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:39.724664 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a29f1-6e9a-457b-b4e4-03d5b5ee80cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:39.725273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a29f1-6e9a-457b-b4e4-03d5b5ee80cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:39.731249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-869a29f1-6e9a-457b-b4e4-03d5b5ee80cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:39.782462 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a29f1-6e9a-457b-b4e4-03d5b5ee80cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:39.791286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a29f1-6e9a-457b-b4e4-03d5b5ee80cd service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:39.840878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 890/1757] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:39.843072 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a29f1-6e9a-457b-b4e4-03d5b5ee80cd service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:39.869258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-869a29f1-6e9a-457b-b4e4-03d5b5ee80cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:39.895142 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-bdb6c610-5f8f-4fa8-bf66-2bc6d55a7952 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72f1f84cc2b44a2ea64303b044f798bf'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:39.926800 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a29f1-6e9a-457b-b4e4-03d5b5ee80cd service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:39.946317 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85907317-995a-4ef0-a14d-37bf0bb3b8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:39.947376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85907317-995a-4ef0-a14d-37bf0bb3b8a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:39.949645 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85907317-995a-4ef0-a14d-37bf0bb3b8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:39.950709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85907317-995a-4ef0-a14d-37bf0bb3b8a4 admin admin] PATH_INFO: `/v3/projects/62da6115a3514aa784192342e1ec0ca7/users/6ff019d9ab3e479a9a51ed2ed8410c63/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:40.002029 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85907317-995a-4ef0-a14d-37bf0bb3b8a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62da6115a3514aa784192342e1ec0ca7, user_id=6ff019d9ab3e479a9a51ed2ed8410c63, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:40.002820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85907317-995a-4ef0-a14d-37bf0bb3b8a4 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:40.024118 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-bdb6c610-5f8f-4fa8-bf66-2bc6d55a7952 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72f1f84cc2b44a2ea64303b044f798bf'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:40.024904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-bdb6c610-5f8f-4fa8-bf66-2bc6d55a7952 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72f1f84cc2b44a2ea64303b044f798bf'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:40.026478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-bdb6c610-5f8f-4fa8-bf66-2bc6d55a7952 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72f1f84cc2b44a2ea64303b044f798bf'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:40.040409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-bdb6c610-5f8f-4fa8-bf66-2bc6d55a7952 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72f1f84cc2b44a2ea64303b044f798bf'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:40.091182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 868/1758] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:36:39 2022] DELETE /identity/v3/users/72f1f84cc2b44a2ea64303b044f798bf => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:40.122678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 891/1759] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:39 2022] PUT /identity/v3/projects/62da6115a3514aa784192342e1ec0ca7/users/6ff019d9ab3e479a9a51ed2ed8410c63/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:40.132359 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6c610-5f8f-4fa8-bf66-2bc6d55a7952 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:40.136184 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-bdb6c610-5f8f-4fa8-bf66-2bc6d55a7952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:40.256478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6c610-5f8f-4fa8-bf66-2bc6d55a7952 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:40.345843 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee49300-31fd-4521-9900-a308441a53a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:40.346588 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee49300-31fd-4521-9900-a308441a53a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:40.346958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee49300-31fd-4521-9900-a308441a53a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:40.347294 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee49300-31fd-4521-9900-a308441a53a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:40.348410 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee49300-31fd-4521-9900-a308441a53a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:40.350392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee49300-31fd-4521-9900-a308441a53a8 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:40.395915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 869/1760] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:40.397973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85907317-995a-4ef0-a14d-37bf0bb3b8a4 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:40.420483 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85907317-995a-4ef0-a14d-37bf0bb3b8a4 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:40.430522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38678698-e2dd-42cb-b5cd-a1609cddd25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:40.431413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38678698-e2dd-42cb-b5cd-a1609cddd25b admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:40.432241 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38678698-e2dd-42cb-b5cd-a1609cddd25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:40.432623 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38678698-e2dd-42cb-b5cd-a1609cddd25b admin admin] PATH_INFO: `/v3/projects/62da6115a3514aa784192342e1ec0ca7/users/6ff019d9ab3e479a9a51ed2ed8410c63/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:40.438496 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38678698-e2dd-42cb-b5cd-a1609cddd25b admin admin] RBAC: Authorizing `identity:create_grant(project_id=62da6115a3514aa784192342e1ec0ca7, user_id=6ff019d9ab3e479a9a51ed2ed8410c63, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:40.452913 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38678698-e2dd-42cb-b5cd-a1609cddd25b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:40.485348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 892/1761] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:40 2022] PUT /identity/v3/projects/62da6115a3514aa784192342e1ec0ca7/users/6ff019d9ab3e479a9a51ed2ed8410c63/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:40.487376 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa95d7-13af-48dc-8c62-4a8c07dde9ef None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:40.487671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa95d7-13af-48dc-8c62-4a8c07dde9ef None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:40.487950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa95d7-13af-48dc-8c62-4a8c07dde9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:40.546163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-ebaa95d7-13af-48dc-8c62-4a8c07dde9ef None None] MFA Rules not processed for user `6ff019d9ab3e479a9a51ed2ed8410c63`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:40.549189 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-ebaa95d7-13af-48dc-8c62-4a8c07dde9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:40.676365 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 870/1762] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:40.687524 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38678698-e2dd-42cb-b5cd-a1609cddd25b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:40.690234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-38678698-e2dd-42cb-b5cd-a1609cddd25b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:40.891201 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:40.965030 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38678698-e2dd-42cb-b5cd-a1609cddd25b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:40.982682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52d368-b430-4d75-9720-f8a893d45124 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:40.982682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52d368-b430-4d75-9720-f8a893d45124 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:40.982682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52d368-b430-4d75-9720-f8a893d45124 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:40.983557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52d368-b430-4d75-9720-f8a893d45124 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:41.026812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52d368-b430-4d75-9720-f8a893d45124 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:41.030543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52d368-b430-4d75-9720-f8a893d45124 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:41.066808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 893/1763] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:40 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:41.501863 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa95d7-13af-48dc-8c62-4a8c07dde9ef None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:41.514353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52d368-b430-4d75-9720-f8a893d45124 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:41.515875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7a52d368-b430-4d75-9720-f8a893d45124 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:41.519409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa95d7-13af-48dc-8c62-4a8c07dde9ef None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:41.523869 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc55467-6487-4d84-b605-4a8e7353d764 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:41.524565 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc55467-6487-4d84-b605-4a8e7353d764 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:41.524843 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc55467-6487-4d84-b605-4a8e7353d764 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:41.525156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc55467-6487-4d84-b605-4a8e7353d764 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:41.528004 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-0bc55467-6487-4d84-b605-4a8e7353d764 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:41.619979 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc55467-6487-4d84-b605-4a8e7353d764 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:41.632029 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc55467-6487-4d84-b605-4a8e7353d764 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:41.634165 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52d368-b430-4d75-9720-f8a893d45124 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:41.636175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 871/1764] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:41.638395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1579be9-1cdc-40b5-a354-971be2c633d9 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:41.638684 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1579be9-1cdc-40b5-a354-971be2c633d9 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:41.638967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1579be9-1cdc-40b5-a354-971be2c633d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:41.639540 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56369d-7f90-43f6-a97e-c665b628faa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:41.640401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56369d-7f90-43f6-a97e-c665b628faa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:41.640872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56369d-7f90-43f6-a97e-c665b628faa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:41.641265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56369d-7f90-43f6-a97e-c665b628faa2 admin admin] PATH_INFO: `/v3/projects/506c707baa5e459b888946eddaa62583` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:41.642960 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a56369d-7f90-43f6-a97e-c665b628faa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=506c707baa5e459b888946eddaa62583)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:41.645160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a56369d-7f90-43f6-a97e-c665b628faa2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:41.661709 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-b1579be9-1cdc-40b5-a354-971be2c633d9 None None] MFA Rules not processed for user `6712334ebb77462daa89be58d4480846`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:41.662939 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b1579be9-1cdc-40b5-a354-971be2c633d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:41.698868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:41.734969 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 872/1765] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:41.800490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6a56369d-7f90-43f6-a97e-c665b628faa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '506c707baa5e459b888946eddaa62583', 'request_id': 'req-6a56369d-7f90-43f6-a97e-c665b628faa2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:41.807281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6a56369d-7f90-43f6-a97e-c665b628faa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '506c707baa5e459b888946eddaa62583', 'request_id': 'req-6a56369d-7f90-43f6-a97e-c665b628faa2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:41.831916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6a56369d-7f90-43f6-a97e-c665b628faa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '506c707baa5e459b888946eddaa62583', 'request_id': 'req-6a56369d-7f90-43f6-a97e-c665b628faa2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:41.894746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6a56369d-7f90-43f6-a97e-c665b628faa2 admin admin] The token cache is being invalidate because project 506c707baa5e459b888946eddaa62583 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:36:41.895190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6a56369d-7f90-43f6-a97e-c665b628faa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:41.895970 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6a56369d-7f90-43f6-a97e-c665b628faa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:41.902746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 894/1766] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:36:41 2022] DELETE /identity/v3/projects/506c707baa5e459b888946eddaa62583 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:36:41.903495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e170c-e0d2-4a58-b593-943f8f89650c None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:41.903787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e170c-e0d2-4a58-b593-943f8f89650c None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:41.904058 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e170c-e0d2-4a58-b593-943f8f89650c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:41.966909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-783e170c-e0d2-4a58-b593-943f8f89650c None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:41.967156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-783e170c-e0d2-4a58-b593-943f8f89650c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:42.137747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:42.226893 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be15a7-5cdb-47f1-8b40-460326595714 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:42.227410 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be15a7-5cdb-47f1-8b40-460326595714 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:42.227688 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be15a7-5cdb-47f1-8b40-460326595714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:42.231025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 873/1767] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:42.353899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b5be15a7-5cdb-47f1-8b40-460326595714 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:42.359837 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b5be15a7-5cdb-47f1-8b40-460326595714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:42.408566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 895/1768] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:36:42.412767 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e170c-e0d2-4a58-b593-943f8f89650c None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:42.431356 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e170c-e0d2-4a58-b593-943f8f89650c None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:42.462086 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bac8fc-3006-45f2-88e8-c85b6bd6fd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:42.462526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bac8fc-3006-45f2-88e8-c85b6bd6fd25 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:42.462806 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bac8fc-3006-45f2-88e8-c85b6bd6fd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:42.463077 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bac8fc-3006-45f2-88e8-c85b6bd6fd25 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:42.463874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bac8fc-3006-45f2-88e8-c85b6bd6fd25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:36:42.464183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bac8fc-3006-45f2-88e8-c85b6bd6fd25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:42.495212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bac8fc-3006-45f2-88e8-c85b6bd6fd25 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:42.573109 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be15a7-5cdb-47f1-8b40-460326595714 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:42.593590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be15a7-5cdb-47f1-8b40-460326595714 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:42.599094 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 874/1769] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:36:42.611998 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c22ec6-1612-4385-956a-391ad4703f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:42.632951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c22ec6-1612-4385-956a-391ad4703f8c admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:42.633366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c22ec6-1612-4385-956a-391ad4703f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:42.633654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c22ec6-1612-4385-956a-391ad4703f8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:42.634367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c22ec6-1612-4385-956a-391ad4703f8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:42.649602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c22ec6-1612-4385-956a-391ad4703f8c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:42.842008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 896/1770] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:42 2022] POST /identity/v3/projects => generated 381 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:36:42.845925 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bac8fc-3006-45f2-88e8-c85b6bd6fd25 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:42.861265 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bac8fc-3006-45f2-88e8-c85b6bd6fd25 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:42.982854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a08e6-1fa9-4ef4-9b07-878bd05e674d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:42.983153 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a08e6-1fa9-4ef4-9b07-878bd05e674d admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:42.985207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a08e6-1fa9-4ef4-9b07-878bd05e674d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:42.985535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a08e6-1fa9-4ef4-9b07-878bd05e674d admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:42.988053 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a08e6-1fa9-4ef4-9b07-878bd05e674d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:42.994147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a08e6-1fa9-4ef4-9b07-878bd05e674d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:43.091902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 875/1771] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:42 2022] POST /identity/v3/users => generated 291 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:43.096154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c22ec6-1612-4385-956a-391ad4703f8c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:43.113199 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c22ec6-1612-4385-956a-391ad4703f8c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:43.125783 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496657f-5b03-4729-9699-22cdf0938d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:43.127122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496657f-5b03-4729-9699-22cdf0938d8f admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:43.127405 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496657f-5b03-4729-9699-22cdf0938d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:43.127900 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496657f-5b03-4729-9699-22cdf0938d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:43.128563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496657f-5b03-4729-9699-22cdf0938d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:43.130324 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496657f-5b03-4729-9699-22cdf0938d8f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:43.174397 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 897/1772] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:36:43.177321 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a08e6-1fa9-4ef4-9b07-878bd05e674d admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:43.194776 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a08e6-1fa9-4ef4-9b07-878bd05e674d admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:43.242091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911f410-823d-4a57-b57a-ea88c7f559fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:43.242742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911f410-823d-4a57-b57a-ea88c7f559fe admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:43.243007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911f410-823d-4a57-b57a-ea88c7f559fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:43.247366 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911f410-823d-4a57-b57a-ea88c7f559fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:43.248164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911f410-823d-4a57-b57a-ea88c7f559fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:43.253417 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911f410-823d-4a57-b57a-ea88c7f559fe admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:43.277468 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 876/1773] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:43.280240 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496657f-5b03-4729-9699-22cdf0938d8f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:43.294860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496657f-5b03-4729-9699-22cdf0938d8f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:43.305413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9756b7-3e71-4378-b98c-c7604af01003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:43.306264 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9756b7-3e71-4378-b98c-c7604af01003 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:43.306602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9756b7-3e71-4378-b98c-c7604af01003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:43.306982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9756b7-3e71-4378-b98c-c7604af01003 admin admin] PATH_INFO: `/v3/projects/88dc91cd104242938a2dec34b6188830/users/5c1fa0a4988c4934a281a2d00880c925/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:43.335740 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9756b7-3e71-4378-b98c-c7604af01003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88dc91cd104242938a2dec34b6188830, user_id=5c1fa0a4988c4934a281a2d00880c925, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:43.340943 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9756b7-3e71-4378-b98c-c7604af01003 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:43.379970 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:43.463346 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 898/1774] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:43 2022] PUT /identity/v3/projects/88dc91cd104242938a2dec34b6188830/users/5c1fa0a4988c4934a281a2d00880c925/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:43.477290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97835375-fb59-4ee6-a2ec-bbccc673e6c8 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:43.477290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97835375-fb59-4ee6-a2ec-bbccc673e6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:43.477290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97835375-fb59-4ee6-a2ec-bbccc673e6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:43.519160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-97835375-fb59-4ee6-a2ec-bbccc673e6c8 None None] MFA Rules not processed for user `5c1fa0a4988c4934a281a2d00880c925`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:43.520554 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-97835375-fb59-4ee6-a2ec-bbccc673e6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:43.679415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9756b7-3e71-4378-b98c-c7604af01003 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:43.693262 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 877/1775] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:43.851523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9756b7-3e71-4378-b98c-c7604af01003 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:43.869708 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b468970-59a6-4e92-aba7-47efc0ad00b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:43.871231 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b468970-59a6-4e92-aba7-47efc0ad00b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:43.871656 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b468970-59a6-4e92-aba7-47efc0ad00b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:43.872069 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b468970-59a6-4e92-aba7-47efc0ad00b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:43.873159 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b468970-59a6-4e92-aba7-47efc0ad00b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:43.876968 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b468970-59a6-4e92-aba7-47efc0ad00b0 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:44.000688 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 899/1776] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:43 2022] POST /identity/v3/projects => generated 379 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:44.006248 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97835375-fb59-4ee6-a2ec-bbccc673e6c8 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:44.024934 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97835375-fb59-4ee6-a2ec-bbccc673e6c8 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:44.175546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3091f5-fb2c-4227-9306-542e68dea94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:44.176529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3091f5-fb2c-4227-9306-542e68dea94d admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:44.177085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3091f5-fb2c-4227-9306-542e68dea94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:44.177605 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3091f5-fb2c-4227-9306-542e68dea94d admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:44.178732 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3091f5-fb2c-4227-9306-542e68dea94d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:44.181267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3091f5-fb2c-4227-9306-542e68dea94d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:44.287527 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 878/1777] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:44 2022] POST /identity/v3/users => generated 296 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:44.290260 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b468970-59a6-4e92-aba7-47efc0ad00b0 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:44.316009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b468970-59a6-4e92-aba7-47efc0ad00b0 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:44.318880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce8446-7a6d-4ea4-bb9d-753231334432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:44.319608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce8446-7a6d-4ea4-bb9d-753231334432 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:44.319982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce8446-7a6d-4ea4-bb9d-753231334432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:44.320348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce8446-7a6d-4ea4-bb9d-753231334432 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:44.321286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce8446-7a6d-4ea4-bb9d-753231334432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:44.323830 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce8446-7a6d-4ea4-bb9d-753231334432 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:44.357259 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 900/1778] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:44.361262 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3091f5-fb2c-4227-9306-542e68dea94d admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:44.374669 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3091f5-fb2c-4227-9306-542e68dea94d admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:44.405106 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4bf94-ee54-4023-aa6d-741ad8131cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:44.409594 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4bf94-ee54-4023-aa6d-741ad8131cf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:44.411757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4bf94-ee54-4023-aa6d-741ad8131cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:44.415260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4bf94-ee54-4023-aa6d-741ad8131cf7 admin admin] PATH_INFO: `/v3/projects/480545e4a7954166b8afe6be2690e1b9/users/3168c1084c9f46ad8d19030bcd1f3bcc/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:44.470235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d4bf94-ee54-4023-aa6d-741ad8131cf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=480545e4a7954166b8afe6be2690e1b9, user_id=3168c1084c9f46ad8d19030bcd1f3bcc, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:44.474850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d4bf94-ee54-4023-aa6d-741ad8131cf7 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:44.587894 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 879/1779] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:44 2022] PUT /identity/v3/projects/480545e4a7954166b8afe6be2690e1b9/users/3168c1084c9f46ad8d19030bcd1f3bcc/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:44.591749 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce8446-7a6d-4ea4-bb9d-753231334432 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:44.606277 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce8446-7a6d-4ea4-bb9d-753231334432 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:44.746346 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5ed28-fe75-45cf-800c-e9b84d545cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:44.747239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5ed28-fe75-45cf-800c-e9b84d545cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:44.747520 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5ed28-fe75-45cf-800c-e9b84d545cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:44.747820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5ed28-fe75-45cf-800c-e9b84d545cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:44.751333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5ed28-fe75-45cf-800c-e9b84d545cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:44.752990 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5ed28-fe75-45cf-800c-e9b84d545cf5 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:44.791693 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 901/1780] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:44.796442 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4bf94-ee54-4023-aa6d-741ad8131cf7 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:44.821319 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4bf94-ee54-4023-aa6d-741ad8131cf7 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:44.836610 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4b9f1-0f92-4389-8027-d7e9c17fed8b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:44.836610 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4b9f1-0f92-4389-8027-d7e9c17fed8b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:44.838229 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4b9f1-0f92-4389-8027-d7e9c17fed8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:44.849998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff106fe-5c82-4c50-a777-41c00cec854c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:44.850980 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff106fe-5c82-4c50-a777-41c00cec854c admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:44.851425 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff106fe-5c82-4c50-a777-41c00cec854c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:44.851864 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff106fe-5c82-4c50-a777-41c00cec854c admin admin] PATH_INFO: `/v3/projects/480545e4a7954166b8afe6be2690e1b9/users/3168c1084c9f46ad8d19030bcd1f3bcc/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:44.854249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff106fe-5c82-4c50-a777-41c00cec854c admin admin] RBAC: Authorizing `identity:create_grant(project_id=480545e4a7954166b8afe6be2690e1b9, user_id=3168c1084c9f46ad8d19030bcd1f3bcc, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:44.858482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff106fe-5c82-4c50-a777-41c00cec854c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:44.897792 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 880/1781] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:44 2022] PUT /identity/v3/projects/480545e4a7954166b8afe6be2690e1b9/users/3168c1084c9f46ad8d19030bcd1f3bcc/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:36:44.906967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f1d4b9f1-0f92-4389-8027-d7e9c17fed8b None None] MFA Rules not processed for user `d4d9bea213fa4401992d8e7894726e05`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:44.906967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f1d4b9f1-0f92-4389-8027-d7e9c17fed8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:44.920913 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52c3c4-4f04-4e7b-8686-78ab5d296f77 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:44.921254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52c3c4-4f04-4e7b-8686-78ab5d296f77 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:44.921496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52c3c4-4f04-4e7b-8686-78ab5d296f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:45.047136 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-8b52c3c4-4f04-4e7b-8686-78ab5d296f77 None None] MFA Rules not processed for user `3168c1084c9f46ad8d19030bcd1f3bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:45.049613 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-8b52c3c4-4f04-4e7b-8686-78ab5d296f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:45.079832 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 902/1782] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:45.102298 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:45.169953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4b9f1-0f92-4389-8027-d7e9c17fed8b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:45.172375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f1d4b9f1-0f92-4389-8027-d7e9c17fed8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:45.199399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 881/1783] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:44 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:45.209454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52c3c4-4f04-4e7b-8686-78ab5d296f77 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:45.212853 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-8b52c3c4-4f04-4e7b-8686-78ab5d296f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:45.302535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52c3c4-4f04-4e7b-8686-78ab5d296f77 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:45.309817 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4b9f1-0f92-4389-8027-d7e9c17fed8b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:45.312341 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327a39a-a695-402f-ac50-1e766e99795d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:45.315782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327a39a-a695-402f-ac50-1e766e99795d service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:45.316273 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327a39a-a695-402f-ac50-1e766e99795d service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:45.316819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327a39a-a695-402f-ac50-1e766e99795d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:45.321161 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca241d-5645-47f5-9088-8a7eedf062a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:45.323700 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca241d-5645-47f5-9088-8a7eedf062a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:45.324138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca241d-5645-47f5-9088-8a7eedf062a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:45.324836 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca241d-5645-47f5-9088-8a7eedf062a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:45.333682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e6ca241d-5645-47f5-9088-8a7eedf062a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:45.336112 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327a39a-a695-402f-ac50-1e766e99795d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:45.336390 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327a39a-a695-402f-ac50-1e766e99795d service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:45.373441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 882/1784] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:45.436148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca241d-5645-47f5-9088-8a7eedf062a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:45.438673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca241d-5645-47f5-9088-8a7eedf062a1 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:45.466203 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:45.476379 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 903/1785] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:45.683530 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327a39a-a695-402f-ac50-1e766e99795d service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:45.696002 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327a39a-a695-402f-ac50-1e766e99795d service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:45.705393 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f6173-b0e4-4b63-a051-bc25c6f63839 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:45.706177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f6173-b0e4-4b63-a051-bc25c6f63839 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:45.707587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f6173-b0e4-4b63-a051-bc25c6f63839 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:45.708816 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f6173-b0e4-4b63-a051-bc25c6f63839 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:45.803458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f6173-b0e4-4b63-a051-bc25c6f63839 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:45.805095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f6173-b0e4-4b63-a051-bc25c6f63839 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:45.824642 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 883/1786] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:46.236413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ea666-1648-469c-bf85-9ab0abb87b18 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:46.236413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ea666-1648-469c-bf85-9ab0abb87b18 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:46.236413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ea666-1648-469c-bf85-9ab0abb87b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:46.325976 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-ac8ea666-1648-469c-bf85-9ab0abb87b18 None None] MFA Rules not processed for user `4ae237601b4a4aab9e969fedb5ee37d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:46.327297 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ac8ea666-1648-469c-bf85-9ab0abb87b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:46.437938 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 904/1787] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 19 10:36:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:46.438227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:46.998840 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f6173-b0e4-4b63-a051-bc25c6f63839 service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:47.000337 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-2c6f6173-b0e4-4b63-a051-bc25c6f63839 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:47.127214 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:47.176832 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f6173-b0e4-4b63-a051-bc25c6f63839 service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:47.188192 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dca05a-d789-4ed6-8924-1d315e114767 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17a8bc6c506c45be897bda23eaab485a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:47.188659 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dca05a-d789-4ed6-8924-1d315e114767 service swift] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:47.188742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dca05a-d789-4ed6-8924-1d315e114767 service swift] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:47.189116 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dca05a-d789-4ed6-8924-1d315e114767 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:47.201093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dca05a-d789-4ed6-8924-1d315e114767 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:47.202749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dca05a-d789-4ed6-8924-1d315e114767 service swift] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:47.243839 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 884/1788] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 19 10:36:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 10:36:47.712144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ea666-1648-469c-bf85-9ab0abb87b18 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:47.718353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ac8ea666-1648-469c-bf85-9ab0abb87b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:48.035872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ea666-1648-469c-bf85-9ab0abb87b18 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:48.069774 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21892a4-8ea0-446d-a025-c4fa402bb320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:48.070453 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21892a4-8ea0-446d-a025-c4fa402bb320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:48.070735 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21892a4-8ea0-446d-a025-c4fa402bb320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:48.071021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21892a4-8ea0-446d-a025-c4fa402bb320 admin admin] PATH_INFO: `/v3/users/02e8375c12cc4c7f8c999eb9c5823c21` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:48.075825 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21892a4-8ea0-446d-a025-c4fa402bb320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02e8375c12cc4c7f8c999eb9c5823c21)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:48.079058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21892a4-8ea0-446d-a025-c4fa402bb320 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:48.358930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f21892a4-8ea0-446d-a025-c4fa402bb320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02e8375c12cc4c7f8c999eb9c5823c21'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:48.359989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f21892a4-8ea0-446d-a025-c4fa402bb320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02e8375c12cc4c7f8c999eb9c5823c21'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:48.433680 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f21892a4-8ea0-446d-a025-c4fa402bb320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02e8375c12cc4c7f8c999eb9c5823c21'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:48.434627 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f21892a4-8ea0-446d-a025-c4fa402bb320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02e8375c12cc4c7f8c999eb9c5823c21'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:48.468416 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f21892a4-8ea0-446d-a025-c4fa402bb320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02e8375c12cc4c7f8c999eb9c5823c21'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:48.497331 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 905/1789] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:36:47 2022] DELETE /identity/v3/users/02e8375c12cc4c7f8c999eb9c5823c21 => generated 0 bytes in 787 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:48.502331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a316f-450f-4cf1-a495-198ca851b320 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:48.502855 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a316f-450f-4cf1-a495-198ca851b320 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:48.502936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a316f-450f-4cf1-a495-198ca851b320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:48.570737 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-f20a316f-450f-4cf1-a495-198ca851b320 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:48.572492 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-f20a316f-450f-4cf1-a495-198ca851b320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:48.722427 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 885/1790] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:36:48.806678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21892a4-8ea0-446d-a025-c4fa402bb320 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:48.809125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f21892a4-8ea0-446d-a025-c4fa402bb320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:49.019806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21892a4-8ea0-446d-a025-c4fa402bb320 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:49.032141 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd8655-ca6f-4c7a-9953-8889b034b6d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:49.032872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd8655-ca6f-4c7a-9953-8889b034b6d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:49.033238 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd8655-ca6f-4c7a-9953-8889b034b6d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:49.033596 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd8655-ca6f-4c7a-9953-8889b034b6d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:49.064218 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd8655-ca6f-4c7a-9953-8889b034b6d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:49.065739 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd8655-ca6f-4c7a-9953-8889b034b6d3 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:49.077566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 906/1791] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:49.451919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb152a25-1c59-4aae-a968-48786dc7c64b None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:49.452653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb152a25-1c59-4aae-a968-48786dc7c64b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:49.453460 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb152a25-1c59-4aae-a968-48786dc7c64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:49.547498 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-eb152a25-1c59-4aae-a968-48786dc7c64b None None] MFA Rules not processed for user `a282e74d177544bbbfba0fdb5e52df60`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:49.551918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-eb152a25-1c59-4aae-a968-48786dc7c64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:49.601706 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 886/1792] 10.222.0.73 () {58 vars in 1015 bytes} [Mon Sep 19 10:36:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:49.635309 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd8655-ca6f-4c7a-9953-8889b034b6d3 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:49.641828 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cfbd8655-ca6f-4c7a-9953-8889b034b6d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:49.674977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd8655-ca6f-4c7a-9953-8889b034b6d3 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:49.683935 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df04a56-7169-426c-aae1-fd1cff927f44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:49.687624 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df04a56-7169-426c-aae1-fd1cff927f44 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:49.687624 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df04a56-7169-426c-aae1-fd1cff927f44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:49.687624 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df04a56-7169-426c-aae1-fd1cff927f44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:49.694350 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df04a56-7169-426c-aae1-fd1cff927f44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:49.715226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df04a56-7169-426c-aae1-fd1cff927f44 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:49.726358 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb152a25-1c59-4aae-a968-48786dc7c64b None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:49.726839 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 907/1793] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:49.739406 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb152a25-1c59-4aae-a968-48786dc7c64b None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:49.746494 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0e297-b0f3-43ea-bcf4-abb07c6c8c60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:49.749421 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0e297-b0f3-43ea-bcf4-abb07c6c8c60 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:49.756695 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0e297-b0f3-43ea-bcf4-abb07c6c8c60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:49.757850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0e297-b0f3-43ea-bcf4-abb07c6c8c60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:49.787706 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0e297-b0f3-43ea-bcf4-abb07c6c8c60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:49.788258 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0e297-b0f3-43ea-bcf4-abb07c6c8c60 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:49.828082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 887/1794] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:50.173485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:50.576560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:51.155667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df04a56-7169-426c-aae1-fd1cff927f44 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:51.155667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7df04a56-7169-426c-aae1-fd1cff927f44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:51.185106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df04a56-7169-426c-aae1-fd1cff927f44 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:51.186353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc15d4-1cef-4e5e-8682-254a9e6e729d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:51.187563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc15d4-1cef-4e5e-8682-254a9e6e729d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:51.188102 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc15d4-1cef-4e5e-8682-254a9e6e729d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:51.188593 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc15d4-1cef-4e5e-8682-254a9e6e729d admin admin] PATH_INFO: `/v3/projects/5bffeada6f994879a6475b975463c0f9` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:51.190093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc15d4-1cef-4e5e-8682-254a9e6e729d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bffeada6f994879a6475b975463c0f9)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:51.193045 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc15d4-1cef-4e5e-8682-254a9e6e729d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:51.444807 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0fbc15d4-1cef-4e5e-8682-254a9e6e729d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bffeada6f994879a6475b975463c0f9', 'request_id': 'req-0fbc15d4-1cef-4e5e-8682-254a9e6e729d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:51.493545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0fbc15d4-1cef-4e5e-8682-254a9e6e729d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bffeada6f994879a6475b975463c0f9', 'request_id': 'req-0fbc15d4-1cef-4e5e-8682-254a9e6e729d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:51.520640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0fbc15d4-1cef-4e5e-8682-254a9e6e729d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bffeada6f994879a6475b975463c0f9', 'request_id': 'req-0fbc15d4-1cef-4e5e-8682-254a9e6e729d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:51.863623 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0fbc15d4-1cef-4e5e-8682-254a9e6e729d admin admin] The token cache is being invalidate because project 5bffeada6f994879a6475b975463c0f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:36:51.865409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0fbc15d4-1cef-4e5e-8682-254a9e6e729d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:51.869174 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0fbc15d4-1cef-4e5e-8682-254a9e6e729d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:51.874193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 908/1795] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:36:51 2022] DELETE /identity/v3/projects/5bffeada6f994879a6475b975463c0f9 => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:51.878265 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0e297-b0f3-43ea-bcf4-abb07c6c8c60 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:51.918716 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0e297-b0f3-43ea-bcf4-abb07c6c8c60 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:52.014280 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe48ef-ca4d-4109-9fbb-764463e77fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:52.015190 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe48ef-ca4d-4109-9fbb-764463e77fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:52.016110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe48ef-ca4d-4109-9fbb-764463e77fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:52.016853 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe48ef-ca4d-4109-9fbb-764463e77fd4 admin admin] PATH_INFO: `/v3/users/6ff019d9ab3e479a9a51ed2ed8410c63` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:52.018634 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe48ef-ca4d-4109-9fbb-764463e77fd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ff019d9ab3e479a9a51ed2ed8410c63)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:52.023199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe48ef-ca4d-4109-9fbb-764463e77fd4 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:52.365786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb0f5a-e0dc-44a2-9b97-2b6ef856812f None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:52.366732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb0f5a-e0dc-44a2-9b97-2b6ef856812f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:52.367170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb0f5a-e0dc-44a2-9b97-2b6ef856812f None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:52.369379 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 909/1796] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 19 10:36:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:36:52.389891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925a66d-c515-4888-922b-4bcfd73a9765 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:52.390605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925a66d-c515-4888-922b-4bcfd73a9765 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:52.391054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925a66d-c515-4888-922b-4bcfd73a9765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:52.435553 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b925a66d-c515-4888-922b-4bcfd73a9765 None None] MFA Rules not processed for user `b4007794a30247c4a318af62d05e3b92`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:52.437527 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b925a66d-c515-4888-922b-4bcfd73a9765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:52.489266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-31fe48ef-ca4d-4109-9fbb-764463e77fd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ff019d9ab3e479a9a51ed2ed8410c63'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:52.540595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 910/1797] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 19 10:36:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:36:52.576204 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925a66d-c515-4888-922b-4bcfd73a9765 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:52.577622 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b925a66d-c515-4888-922b-4bcfd73a9765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:52.617332 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925a66d-c515-4888-922b-4bcfd73a9765 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:52.628939 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9fa55-04a0-4ec6-973f-07d4649bf985 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:52.664120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9fa55-04a0-4ec6-973f-07d4649bf985 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:52.664120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9fa55-04a0-4ec6-973f-07d4649bf985 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:52.664120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9fa55-04a0-4ec6-973f-07d4649bf985 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:52.664120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9fa55-04a0-4ec6-973f-07d4649bf985 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:52.664120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9fa55-04a0-4ec6-973f-07d4649bf985 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:52.664120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 911/1798] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:36:52.724720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-31fe48ef-ca4d-4109-9fbb-764463e77fd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ff019d9ab3e479a9a51ed2ed8410c63'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:52.730447 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-31fe48ef-ca4d-4109-9fbb-764463e77fd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ff019d9ab3e479a9a51ed2ed8410c63'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:52.731480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-31fe48ef-ca4d-4109-9fbb-764463e77fd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ff019d9ab3e479a9a51ed2ed8410c63'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:52.740645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-31fe48ef-ca4d-4109-9fbb-764463e77fd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ff019d9ab3e479a9a51ed2ed8410c63'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:52.799393 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 888/1799] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:36:51 2022] DELETE /identity/v3/users/6ff019d9ab3e479a9a51ed2ed8410c63 => generated 0 bytes in 922 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:53.238677 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9fa55-04a0-4ec6-973f-07d4649bf985 service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:53.241310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e9a9fa55-04a0-4ec6-973f-07d4649bf985 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:53.263910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9fa55-04a0-4ec6-973f-07d4649bf985 service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:53.362593 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8c0a4-1894-419d-82e1-0951855bb542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:53.363252 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8c0a4-1894-419d-82e1-0951855bb542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:53.363535 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8c0a4-1894-419d-82e1-0951855bb542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:53.363814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8c0a4-1894-419d-82e1-0951855bb542 admin admin] PATH_INFO: `/v3/users/0a175b8016544995be2ebc976ac058ed` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:53.365148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8c0a4-1894-419d-82e1-0951855bb542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a175b8016544995be2ebc976ac058ed)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:53.372222 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8c0a4-1894-419d-82e1-0951855bb542 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:53.688979 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c0d8c0a4-1894-419d-82e1-0951855bb542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a175b8016544995be2ebc976ac058ed'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:53.689872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c0d8c0a4-1894-419d-82e1-0951855bb542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a175b8016544995be2ebc976ac058ed'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:53.826068 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c0d8c0a4-1894-419d-82e1-0951855bb542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a175b8016544995be2ebc976ac058ed'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:53.829915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c0d8c0a4-1894-419d-82e1-0951855bb542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a175b8016544995be2ebc976ac058ed'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:53.842240 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c0d8c0a4-1894-419d-82e1-0951855bb542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a175b8016544995be2ebc976ac058ed'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:53.861160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 912/1800] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:36:53 2022] DELETE /identity/v3/users/0a175b8016544995be2ebc976ac058ed => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:53.863337 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202f989-a506-4827-b1db-0e666ff03555 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:53.864233 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202f989-a506-4827-b1db-0e666ff03555 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:53.869353 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202f989-a506-4827-b1db-0e666ff03555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:53.890386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-e202f989-a506-4827-b1db-0e666ff03555 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:53.892654 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e202f989-a506-4827-b1db-0e666ff03555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:53.961879 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 889/1801] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:53.970171 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8c0a4-1894-419d-82e1-0951855bb542 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:53.971832 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c0d8c0a4-1894-419d-82e1-0951855bb542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:54.022244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202f989-a506-4827-b1db-0e666ff03555 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:54.023912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e202f989-a506-4827-b1db-0e666ff03555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:54.045130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202f989-a506-4827-b1db-0e666ff03555 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:54.053198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0af6c6-3011-4a5f-8c38-62da224cb2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:54.053198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0af6c6-3011-4a5f-8c38-62da224cb2b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:54.053198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0af6c6-3011-4a5f-8c38-62da224cb2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:54.053198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0af6c6-3011-4a5f-8c38-62da224cb2b7 admin admin] PATH_INFO: `/v3/projects/62da6115a3514aa784192342e1ec0ca7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:54.053999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0af6c6-3011-4a5f-8c38-62da224cb2b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62da6115a3514aa784192342e1ec0ca7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:54.056275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8c0a4-1894-419d-82e1-0951855bb542 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:54.056428 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0af6c6-3011-4a5f-8c38-62da224cb2b7 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:54.060642 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507ef87-3281-40a0-8e2e-91620c4a0929 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:54.061303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507ef87-3281-40a0-8e2e-91620c4a0929 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:54.061571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507ef87-3281-40a0-8e2e-91620c4a0929 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:54.061841 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507ef87-3281-40a0-8e2e-91620c4a0929 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:54.072328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507ef87-3281-40a0-8e2e-91620c4a0929 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:54.074882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507ef87-3281-40a0-8e2e-91620c4a0929 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:54.091048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 913/1802] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:54.092516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4b693-0599-4b57-9e21-cbda01100dac None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:54.092842 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4b693-0599-4b57-9e21-cbda01100dac None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:54.093265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4b693-0599-4b57-9e21-cbda01100dac None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:54.094678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 914/1803] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 19 10:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:36:54.107403 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263c01f-0469-44a0-92e8-55d442c3846b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:54.107765 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263c01f-0469-44a0-92e8-55d442c3846b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:54.108051 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263c01f-0469-44a0-92e8-55d442c3846b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:54.128742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-0263c01f-0469-44a0-92e8-55d442c3846b None None] MFA Rules not processed for user `b4007794a30247c4a318af62d05e3b92`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:54.130001 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0263c01f-0469-44a0-92e8-55d442c3846b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:54.205224 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-6b0af6c6-3011-4a5f-8c38-62da224cb2b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62da6115a3514aa784192342e1ec0ca7', 'request_id': 'req-6b0af6c6-3011-4a5f-8c38-62da224cb2b7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:54.214445 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-6b0af6c6-3011-4a5f-8c38-62da224cb2b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62da6115a3514aa784192342e1ec0ca7', 'request_id': 'req-6b0af6c6-3011-4a5f-8c38-62da224cb2b7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:54.221440 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 915/1804] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 19 10:36:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:36:54.222553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-6b0af6c6-3011-4a5f-8c38-62da224cb2b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62da6115a3514aa784192342e1ec0ca7', 'request_id': 'req-6b0af6c6-3011-4a5f-8c38-62da224cb2b7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:54.241404 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263c01f-0469-44a0-92e8-55d442c3846b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:54.243389 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0263c01f-0469-44a0-92e8-55d442c3846b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:54.258522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263c01f-0469-44a0-92e8-55d442c3846b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:54.289139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-6b0af6c6-3011-4a5f-8c38-62da224cb2b7 admin admin] The token cache is being invalidate because project 62da6115a3514aa784192342e1ec0ca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:36:54.289139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-6b0af6c6-3011-4a5f-8c38-62da224cb2b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:54.289788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-6b0af6c6-3011-4a5f-8c38-62da224cb2b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:54.293189 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 890/1805] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:36:54 2022] DELETE /identity/v3/projects/62da6115a3514aa784192342e1ec0ca7 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:54.298304 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0f221-7725-47f6-88df-cbb483dfbc38 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:54.298812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0f221-7725-47f6-88df-cbb483dfbc38 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:54.299252 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0f221-7725-47f6-88df-cbb483dfbc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:54.325267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-45e0f221-7725-47f6-88df-cbb483dfbc38 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:54.326785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-45e0f221-7725-47f6-88df-cbb483dfbc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:54.345258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68abac-8975-4832-bd08-79c95bf0b864 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:54.345894 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68abac-8975-4832-bd08-79c95bf0b864 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:54.346243 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68abac-8975-4832-bd08-79c95bf0b864 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:54.346588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68abac-8975-4832-bd08-79c95bf0b864 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:54.348691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bd68abac-8975-4832-bd08-79c95bf0b864 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:54.373869 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68abac-8975-4832-bd08-79c95bf0b864 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:54.376601 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68abac-8975-4832-bd08-79c95bf0b864 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:54.393489 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:54.405267 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 916/1806] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:54.441162 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 891/1807] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:54.442705 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806ce1f-8653-408d-8364-8469adce2e52 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:54.443001 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806ce1f-8653-408d-8364-8469adce2e52 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:54.443281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806ce1f-8653-408d-8364-8469adce2e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:54.475161 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-0806ce1f-8653-408d-8364-8469adce2e52 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:54.476482 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0806ce1f-8653-408d-8364-8469adce2e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:54.496045 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 917/1808] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:54.496294 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0f221-7725-47f6-88df-cbb483dfbc38 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:54.513190 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0f221-7725-47f6-88df-cbb483dfbc38 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:54.523373 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bdef9-d756-426b-bbf3-79e3dd202c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:54.525813 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bdef9-d756-426b-bbf3-79e3dd202c20 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:54.528006 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bdef9-d756-426b-bbf3-79e3dd202c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:54.528411 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bdef9-d756-426b-bbf3-79e3dd202c20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:54.530702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bdef9-d756-426b-bbf3-79e3dd202c20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:36:54.531129 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bdef9-d756-426b-bbf3-79e3dd202c20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:54.535845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bdef9-d756-426b-bbf3-79e3dd202c20 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:54.555317 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 892/1809] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:36:54.560670 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806ce1f-8653-408d-8364-8469adce2e52 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:54.570442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806ce1f-8653-408d-8364-8469adce2e52 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:54.582138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb21e40-a354-47c9-a7ca-bbdfed69db24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:54.582902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb21e40-a354-47c9-a7ca-bbdfed69db24 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:54.583414 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb21e40-a354-47c9-a7ca-bbdfed69db24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:54.583892 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb21e40-a354-47c9-a7ca-bbdfed69db24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:54.585688 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb21e40-a354-47c9-a7ca-bbdfed69db24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:54.587818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb21e40-a354-47c9-a7ca-bbdfed69db24 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:54.711114 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 918/1810] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:54 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:36:54.712363 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bdef9-d756-426b-bbf3-79e3dd202c20 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:54.729797 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bdef9-d756-426b-bbf3-79e3dd202c20 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:54.780578 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:54.791843 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ba203-037f-49dc-b29a-073ddbd34999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:54.792460 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ba203-037f-49dc-b29a-073ddbd34999 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:54.792740 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ba203-037f-49dc-b29a-073ddbd34999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:54.793014 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ba203-037f-49dc-b29a-073ddbd34999 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:54.793967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ba203-037f-49dc-b29a-073ddbd34999 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:54.795898 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ba203-037f-49dc-b29a-073ddbd34999 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:54.835208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb21e40-a354-47c9-a7ca-bbdfed69db24 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:54.837089 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-eeb21e40-a354-47c9-a7ca-bbdfed69db24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:54.849971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb21e40-a354-47c9-a7ca-bbdfed69db24 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:54.854675 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0ac2e-befe-4a50-a7d5-db665e3f89bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:54.855327 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0ac2e-befe-4a50-a7d5-db665e3f89bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:54.855607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0ac2e-befe-4a50-a7d5-db665e3f89bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:54.855883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0ac2e-befe-4a50-a7d5-db665e3f89bf admin admin] PATH_INFO: `/v3/projects/ff7bba98ba344851a48ae64840a7106c` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:54.857381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb0ac2e-befe-4a50-a7d5-db665e3f89bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff7bba98ba344851a48ae64840a7106c)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:54.859288 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb0ac2e-befe-4a50-a7d5-db665e3f89bf admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:54.894023 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 893/1811] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:54 2022] POST /identity/v3/users => generated 294 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:36:54.897975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ba203-037f-49dc-b29a-073ddbd34999 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:54.909764 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ba203-037f-49dc-b29a-073ddbd34999 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:54.914418 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603aea80-74b5-44ee-a3ae-7ba81d92c39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:54.915377 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603aea80-74b5-44ee-a3ae-7ba81d92c39b admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:54.915653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603aea80-74b5-44ee-a3ae-7ba81d92c39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:54.916110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603aea80-74b5-44ee-a3ae-7ba81d92c39b admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:54.916902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603aea80-74b5-44ee-a3ae-7ba81d92c39b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:54.918522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603aea80-74b5-44ee-a3ae-7ba81d92c39b admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:54.930402 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 894/1812] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:54.933695 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603aea80-74b5-44ee-a3ae-7ba81d92c39b admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:54.943122 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603aea80-74b5-44ee-a3ae-7ba81d92c39b admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:54.947647 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7ccee-de48-4848-b46c-cd914466383f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:54.948742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7ccee-de48-4848-b46c-cd914466383f admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:54.949016 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7ccee-de48-4848-b46c-cd914466383f admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:54.949310 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7ccee-de48-4848-b46c-cd914466383f admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:54.949933 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7ccee-de48-4848-b46c-cd914466383f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:54.951495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7ccee-de48-4848-b46c-cd914466383f admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:54.962728 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 895/1813] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:36:54.968351 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7ccee-de48-4848-b46c-cd914466383f admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:54.970142 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-beb0ac2e-befe-4a50-a7d5-db665e3f89bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff7bba98ba344851a48ae64840a7106c', 'request_id': 'req-beb0ac2e-befe-4a50-a7d5-db665e3f89bf'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:54.976299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-beb0ac2e-befe-4a50-a7d5-db665e3f89bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff7bba98ba344851a48ae64840a7106c', 'request_id': 'req-beb0ac2e-befe-4a50-a7d5-db665e3f89bf'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:54.979856 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7ccee-de48-4848-b46c-cd914466383f admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:54.983118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-beb0ac2e-befe-4a50-a7d5-db665e3f89bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff7bba98ba344851a48ae64840a7106c', 'request_id': 'req-beb0ac2e-befe-4a50-a7d5-db665e3f89bf'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:55.014251 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:55.038819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ca6cc-67eb-4772-be61-dae270fe575e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:55.039569 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ca6cc-67eb-4772-be61-dae270fe575e admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:55.039797 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ca6cc-67eb-4772-be61-dae270fe575e admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:55.040085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ca6cc-67eb-4772-be61-dae270fe575e admin admin] PATH_INFO: `/v3/projects/4ba58ccb64164ca49cf7c7cc89fbc15d/users/30abf9f80bcb425aa6e1e4c08df55050/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:55.055015 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ca6cc-67eb-4772-be61-dae270fe575e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ba58ccb64164ca49cf7c7cc89fbc15d, user_id=30abf9f80bcb425aa6e1e4c08df55050, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:55.058494 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ca6cc-67eb-4772-be61-dae270fe575e admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:55.078885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-beb0ac2e-befe-4a50-a7d5-db665e3f89bf admin admin] The token cache is being invalidate because project ff7bba98ba344851a48ae64840a7106c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:36:55.078885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-beb0ac2e-befe-4a50-a7d5-db665e3f89bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:55.078885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-beb0ac2e-befe-4a50-a7d5-db665e3f89bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:55.078885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 919/1814] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:36:54 2022] DELETE /identity/v3/projects/ff7bba98ba344851a48ae64840a7106c => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:36:55.082396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0ac2e-befe-4a50-a7d5-db665e3f89bf admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:55.085201 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-beb0ac2e-befe-4a50-a7d5-db665e3f89bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:55.097904 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0ac2e-befe-4a50-a7d5-db665e3f89bf admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:55.103468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58183322-89ee-421b-a06c-ee06a9d34a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:55.106022 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58183322-89ee-421b-a06c-ee06a9d34a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:55.106439 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58183322-89ee-421b-a06c-ee06a9d34a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:55.106670 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58183322-89ee-421b-a06c-ee06a9d34a1d admin admin] PATH_INFO: `/v3/users/079f07a5d8cd43ab9b26441e93a8c9a3` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:55.109183 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58183322-89ee-421b-a06c-ee06a9d34a1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=079f07a5d8cd43ab9b26441e93a8c9a3)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:55.111101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58183322-89ee-421b-a06c-ee06a9d34a1d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:55.170183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 896/1815] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:54 2022] PUT /identity/v3/projects/4ba58ccb64164ca49cf7c7cc89fbc15d/users/30abf9f80bcb425aa6e1e4c08df55050/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:55.180759 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b04086-be1d-4f5e-8f17-df8a573515bc None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:55.181163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b04086-be1d-4f5e-8f17-df8a573515bc None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:55.181449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b04086-be1d-4f5e-8f17-df8a573515bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:55.208862 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-a8b04086-be1d-4f5e-8f17-df8a573515bc None None] MFA Rules not processed for user `30abf9f80bcb425aa6e1e4c08df55050`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:55.210113 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-a8b04086-be1d-4f5e-8f17-df8a573515bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:55.274022 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 897/1816] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:36:55.279436 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b04086-be1d-4f5e-8f17-df8a573515bc None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:55.280824 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-a8b04086-be1d-4f5e-8f17-df8a573515bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:55.301301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-58183322-89ee-421b-a06c-ee06a9d34a1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '079f07a5d8cd43ab9b26441e93a8c9a3'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:55.302101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-58183322-89ee-421b-a06c-ee06a9d34a1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '079f07a5d8cd43ab9b26441e93a8c9a3'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:55.347703 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b04086-be1d-4f5e-8f17-df8a573515bc None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:55.352318 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ea534-8ed3-494d-8c47-df50ad988c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:55.352869 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ea534-8ed3-494d-8c47-df50ad988c86 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:55.353201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ea534-8ed3-494d-8c47-df50ad988c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:55.353512 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ea534-8ed3-494d-8c47-df50ad988c86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:55.354149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ea534-8ed3-494d-8c47-df50ad988c86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:55.356145 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ea534-8ed3-494d-8c47-df50ad988c86 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:55.377872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-58183322-89ee-421b-a06c-ee06a9d34a1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '079f07a5d8cd43ab9b26441e93a8c9a3'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:55.378824 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-58183322-89ee-421b-a06c-ee06a9d34a1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '079f07a5d8cd43ab9b26441e93a8c9a3'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:55.392054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-58183322-89ee-421b-a06c-ee06a9d34a1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '079f07a5d8cd43ab9b26441e93a8c9a3'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:55.404938 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 920/1817] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:36:55 2022] DELETE /identity/v3/users/079f07a5d8cd43ab9b26441e93a8c9a3 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:36:55.462492 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 898/1818] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:36:55 2022] POST /identity/v3/projects => generated 387 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:36:55.468205 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58183322-89ee-421b-a06c-ee06a9d34a1d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:55.470631 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-58183322-89ee-421b-a06c-ee06a9d34a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:55.535741 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58183322-89ee-421b-a06c-ee06a9d34a1d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:55.590383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73411281-b58e-4efa-b61c-bfa87f8c5df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:55.590995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73411281-b58e-4efa-b61c-bfa87f8c5df5 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:55.591273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73411281-b58e-4efa-b61c-bfa87f8c5df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:55.591540 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73411281-b58e-4efa-b61c-bfa87f8c5df5 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:55.592171 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73411281-b58e-4efa-b61c-bfa87f8c5df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:55.594114 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73411281-b58e-4efa-b61c-bfa87f8c5df5 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:55.677772 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 921/1819] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:36:55 2022] POST /identity/v3/users => generated 300 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:36:55.681871 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ea534-8ed3-494d-8c47-df50ad988c86 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:55.692280 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ea534-8ed3-494d-8c47-df50ad988c86 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:55.697300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214c97f-b946-4bea-b8eb-7d2b0b8a0538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:55.697915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214c97f-b946-4bea-b8eb-7d2b0b8a0538 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:55.698199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214c97f-b946-4bea-b8eb-7d2b0b8a0538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:55.699352 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214c97f-b946-4bea-b8eb-7d2b0b8a0538 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:55.700058 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214c97f-b946-4bea-b8eb-7d2b0b8a0538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:55.701768 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214c97f-b946-4bea-b8eb-7d2b0b8a0538 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:55.712877 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 899/1820] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:55.717910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73411281-b58e-4efa-b61c-bfa87f8c5df5 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:55.727998 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73411281-b58e-4efa-b61c-bfa87f8c5df5 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:55.738313 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f9fef-d9d5-479b-ac76-8c57f1e683cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:55.739175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f9fef-d9d5-479b-ac76-8c57f1e683cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:55.739601 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f9fef-d9d5-479b-ac76-8c57f1e683cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:55.740019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f9fef-d9d5-479b-ac76-8c57f1e683cb admin admin] PATH_INFO: `/v3/projects/6f84f2beacce4ed484a8e67c2d4c993e/users/6ca1a1d2c19b40828b4416c740db021f/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:55.768048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f9fef-d9d5-479b-ac76-8c57f1e683cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f84f2beacce4ed484a8e67c2d4c993e, user_id=6ca1a1d2c19b40828b4416c740db021f, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:55.772388 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f9fef-d9d5-479b-ac76-8c57f1e683cb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:55.839068 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 922/1821] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:55 2022] PUT /identity/v3/projects/6f84f2beacce4ed484a8e67c2d4c993e/users/6ca1a1d2c19b40828b4416c740db021f/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:55.843115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214c97f-b946-4bea-b8eb-7d2b0b8a0538 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:55.854635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214c97f-b946-4bea-b8eb-7d2b0b8a0538 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:55.917979 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15828fd9-0b88-461d-8074-51c8d1be28b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:55.918663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15828fd9-0b88-461d-8074-51c8d1be28b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:55.919174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15828fd9-0b88-461d-8074-51c8d1be28b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:55.919608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15828fd9-0b88-461d-8074-51c8d1be28b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:55.920397 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15828fd9-0b88-461d-8074-51c8d1be28b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:55.922199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15828fd9-0b88-461d-8074-51c8d1be28b1 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:55.933709 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 900/1822] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:36:55.938288 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f9fef-d9d5-479b-ac76-8c57f1e683cb admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:55.951719 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f9fef-d9d5-479b-ac76-8c57f1e683cb admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:55.959077 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8acf9-3464-49af-a37d-e5ba7cb05ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:55.960068 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8acf9-3464-49af-a37d-e5ba7cb05ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:55.960504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8acf9-3464-49af-a37d-e5ba7cb05ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:55.961504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8acf9-3464-49af-a37d-e5ba7cb05ab2 admin admin] PATH_INFO: `/v3/projects/6f84f2beacce4ed484a8e67c2d4c993e/users/6ca1a1d2c19b40828b4416c740db021f/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:55.963510 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8acf9-3464-49af-a37d-e5ba7cb05ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f84f2beacce4ed484a8e67c2d4c993e, user_id=6ca1a1d2c19b40828b4416c740db021f, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:55.966240 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8acf9-3464-49af-a37d-e5ba7cb05ab2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:55.985350 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 923/1823] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:36:55 2022] PUT /identity/v3/projects/6f84f2beacce4ed484a8e67c2d4c993e/users/6ca1a1d2c19b40828b4416c740db021f/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:36:56.004253 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f97461-8ece-4736-b29b-553193f86f9c None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:56.004253 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f97461-8ece-4736-b29b-553193f86f9c None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:56.004253 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f97461-8ece-4736-b29b-553193f86f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:56.043166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-c6f97461-8ece-4736-b29b-553193f86f9c None None] MFA Rules not processed for user `6ca1a1d2c19b40828b4416c740db021f`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:56.044644 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c6f97461-8ece-4736-b29b-553193f86f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:56.117545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8acf9-3464-49af-a37d-e5ba7cb05ab2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:56.120460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-58c8acf9-3464-49af-a37d-e5ba7cb05ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:56.189667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 901/1824] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:55 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:56.202205 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f97461-8ece-4736-b29b-553193f86f9c None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:56.208418 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c6f97461-8ece-4736-b29b-553193f86f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:56.311497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8acf9-3464-49af-a37d-e5ba7cb05ab2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:56.318308 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c3b6b-5f11-45a7-807d-95825821a80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:56.319075 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c3b6b-5f11-45a7-807d-95825821a80f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:56.319411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c3b6b-5f11-45a7-807d-95825821a80f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:56.319756 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c3b6b-5f11-45a7-807d-95825821a80f admin admin] PATH_INFO: `/v3/projects/0d9786b577494ccf859f6585f119a9bc` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:56.321124 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c3b6b-5f11-45a7-807d-95825821a80f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d9786b577494ccf859f6585f119a9bc)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:56.323845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c3b6b-5f11-45a7-807d-95825821a80f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:56.324303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f97461-8ece-4736-b29b-553193f86f9c None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:56.328404 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87282f-19e8-4022-b98e-08d724d0fcba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:56.329296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87282f-19e8-4022-b98e-08d724d0fcba service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:56.329637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87282f-19e8-4022-b98e-08d724d0fcba service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:56.329995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87282f-19e8-4022-b98e-08d724d0fcba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:56.350083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87282f-19e8-4022-b98e-08d724d0fcba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:56.351605 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87282f-19e8-4022-b98e-08d724d0fcba service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:56.362808 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 902/1825] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:56 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:36:56.399739 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87282f-19e8-4022-b98e-08d724d0fcba service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:56.414014 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87282f-19e8-4022-b98e-08d724d0fcba service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:56.424912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03783dfe-642d-410e-aff3-b5cc27fbf4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:56.426279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03783dfe-642d-410e-aff3-b5cc27fbf4cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:56.427071 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03783dfe-642d-410e-aff3-b5cc27fbf4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:56.427609 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03783dfe-642d-410e-aff3-b5cc27fbf4cb admin admin] PATH_INFO: `/v3/users/30abf9f80bcb425aa6e1e4c08df55050` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:56.429200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03783dfe-642d-410e-aff3-b5cc27fbf4cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=30abf9f80bcb425aa6e1e4c08df55050)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:56.431242 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03783dfe-642d-410e-aff3-b5cc27fbf4cb admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:56.740750 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:57.146596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-03783dfe-642d-410e-aff3-b5cc27fbf4cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30abf9f80bcb425aa6e1e4c08df55050'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:57.165926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-aa1c3b6b-5f11-45a7-807d-95825821a80f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d9786b577494ccf859f6585f119a9bc', 'request_id': 'req-aa1c3b6b-5f11-45a7-807d-95825821a80f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:57.184749 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-aa1c3b6b-5f11-45a7-807d-95825821a80f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d9786b577494ccf859f6585f119a9bc', 'request_id': 'req-aa1c3b6b-5f11-45a7-807d-95825821a80f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:57.193326 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-aa1c3b6b-5f11-45a7-807d-95825821a80f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d9786b577494ccf859f6585f119a9bc', 'request_id': 'req-aa1c3b6b-5f11-45a7-807d-95825821a80f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:57.255121 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-03783dfe-642d-410e-aff3-b5cc27fbf4cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30abf9f80bcb425aa6e1e4c08df55050'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:57.257808 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-03783dfe-642d-410e-aff3-b5cc27fbf4cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30abf9f80bcb425aa6e1e4c08df55050'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:57.257808 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-03783dfe-642d-410e-aff3-b5cc27fbf4cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30abf9f80bcb425aa6e1e4c08df55050'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:57.269165 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-03783dfe-642d-410e-aff3-b5cc27fbf4cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30abf9f80bcb425aa6e1e4c08df55050'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:57.333897 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 903/1826] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:36:56 2022] DELETE /identity/v3/users/30abf9f80bcb425aa6e1e4c08df55050 => generated 0 bytes in 931 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:57.335379 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36172e8c-0c61-4f66-8a42-38ad31c6691e None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:57.335671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36172e8c-0c61-4f66-8a42-38ad31c6691e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:57.335946 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36172e8c-0c61-4f66-8a42-38ad31c6691e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:57.377610 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-36172e8c-0c61-4f66-8a42-38ad31c6691e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:57.382637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-36172e8c-0c61-4f66-8a42-38ad31c6691e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:57.402959 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-aa1c3b6b-5f11-45a7-807d-95825821a80f admin admin] The token cache is being invalidate because project 0d9786b577494ccf859f6585f119a9bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:36:57.404100 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-aa1c3b6b-5f11-45a7-807d-95825821a80f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:57.405440 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-aa1c3b6b-5f11-45a7-807d-95825821a80f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:57.410874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 924/1827] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:36:56 2022] DELETE /identity/v3/projects/0d9786b577494ccf859f6585f119a9bc => generated 0 bytes in 1292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:36:57.413845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c3b6b-5f11-45a7-807d-95825821a80f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:57.421512 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-aa1c3b6b-5f11-45a7-807d-95825821a80f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:57.534320 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 904/1828] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:57.554690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36172e8c-0c61-4f66-8a42-38ad31c6691e None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:57.558198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-36172e8c-0c61-4f66-8a42-38ad31c6691e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:57.580296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c3b6b-5f11-45a7-807d-95825821a80f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:57.593509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22c0f2-fc4a-4827-9f57-2bda21ac5ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:57.594954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22c0f2-fc4a-4827-9f57-2bda21ac5ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:57.595521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22c0f2-fc4a-4827-9f57-2bda21ac5ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:57.597764 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22c0f2-fc4a-4827-9f57-2bda21ac5ac9 admin admin] PATH_INFO: `/v3/users/e8058e72bac2463694f49fc6db81ac30` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:57.603443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22c0f2-fc4a-4827-9f57-2bda21ac5ac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8058e72bac2463694f49fc6db81ac30)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:57.610288 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22c0f2-fc4a-4827-9f57-2bda21ac5ac9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:57.681470 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36172e8c-0c61-4f66-8a42-38ad31c6691e None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:57.686128 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b8630-0e63-4738-9c4d-8e17a538ba28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:57.686884 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b8630-0e63-4738-9c4d-8e17a538ba28 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:57.687190 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b8630-0e63-4738-9c4d-8e17a538ba28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:57.687435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b8630-0e63-4738-9c4d-8e17a538ba28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:57.706073 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b8630-0e63-4738-9c4d-8e17a538ba28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:57.708024 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b8630-0e63-4738-9c4d-8e17a538ba28 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:57.728451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 905/1829] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:36:57.920486 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3c22c0f2-fc4a-4827-9f57-2bda21ac5ac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8058e72bac2463694f49fc6db81ac30'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:57.922343 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3c22c0f2-fc4a-4827-9f57-2bda21ac5ac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8058e72bac2463694f49fc6db81ac30'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:58.163501 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3c22c0f2-fc4a-4827-9f57-2bda21ac5ac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8058e72bac2463694f49fc6db81ac30'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:58.163501 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3c22c0f2-fc4a-4827-9f57-2bda21ac5ac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8058e72bac2463694f49fc6db81ac30'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:58.181613 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3c22c0f2-fc4a-4827-9f57-2bda21ac5ac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8058e72bac2463694f49fc6db81ac30'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:58.198628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b8630-0e63-4738-9c4d-8e17a538ba28 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:58.198628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-a38b8630-0e63-4738-9c4d-8e17a538ba28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:58.220645 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 925/1830] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:36:57 2022] DELETE /identity/v3/users/e8058e72bac2463694f49fc6db81ac30 => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:36:58.220800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b8630-0e63-4738-9c4d-8e17a538ba28 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:58.368857 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e57fa-3464-4a31-929d-73e5a5c0ad2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:58.368857 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e57fa-3464-4a31-929d-73e5a5c0ad2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:58.368857 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e57fa-3464-4a31-929d-73e5a5c0ad2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:58.368857 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e57fa-3464-4a31-929d-73e5a5c0ad2b admin admin] PATH_INFO: `/v3/users/6e439fb618af4404b9fbafbad6b996f9` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:58.369332 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e57fa-3464-4a31-929d-73e5a5c0ad2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e439fb618af4404b9fbafbad6b996f9)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:58.369332 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e57fa-3464-4a31-929d-73e5a5c0ad2b admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:58.635262 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-917e57fa-3464-4a31-929d-73e5a5c0ad2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e439fb618af4404b9fbafbad6b996f9'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:58.886964 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-917e57fa-3464-4a31-929d-73e5a5c0ad2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e439fb618af4404b9fbafbad6b996f9'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:58.900375 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-917e57fa-3464-4a31-929d-73e5a5c0ad2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e439fb618af4404b9fbafbad6b996f9'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:58.901919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-917e57fa-3464-4a31-929d-73e5a5c0ad2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e439fb618af4404b9fbafbad6b996f9'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:58.916698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-917e57fa-3464-4a31-929d-73e5a5c0ad2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e439fb618af4404b9fbafbad6b996f9'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:36:58.977703 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891edf68-8339-4924-a2b0-6ccc1a9dbf17 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:58.978310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891edf68-8339-4924-a2b0-6ccc1a9dbf17 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:58.978802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891edf68-8339-4924-a2b0-6ccc1a9dbf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:58.982575 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 906/1831] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:36:58 2022] DELETE /identity/v3/users/6e439fb618af4404b9fbafbad6b996f9 => generated 0 bytes in 777 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:36:59.018663 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-891edf68-8339-4924-a2b0-6ccc1a9dbf17 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:36:59.019970 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-891edf68-8339-4924-a2b0-6ccc1a9dbf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:59.141844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 926/1832] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:36:59.149846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:36:59.232641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e57fa-3464-4a31-929d-73e5a5c0ad2b admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:59.234783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-917e57fa-3464-4a31-929d-73e5a5c0ad2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:59.382365 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e57fa-3464-4a31-929d-73e5a5c0ad2b admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:59.393597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba0727-02a0-4d7e-a45b-59e498f236ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:59.396576 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba0727-02a0-4d7e-a45b-59e498f236ca service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:59.400014 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba0727-02a0-4d7e-a45b-59e498f236ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:59.400394 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba0727-02a0-4d7e-a45b-59e498f236ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:59.433564 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba0727-02a0-4d7e-a45b-59e498f236ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:59.438282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba0727-02a0-4d7e-a45b-59e498f236ca service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:36:59.481633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 907/1833] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:36:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:36:59.725484 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891edf68-8339-4924-a2b0-6ccc1a9dbf17 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:36:59.726579 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-891edf68-8339-4924-a2b0-6ccc1a9dbf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:36:59.751408 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891edf68-8339-4924-a2b0-6ccc1a9dbf17 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:36:59.756949 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81f4d7-12ed-4a98-849c-dddbe1faf979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:36:59.761523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81f4d7-12ed-4a98-849c-dddbe1faf979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:36:59.762129 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81f4d7-12ed-4a98-849c-dddbe1faf979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:36:59.764005 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81f4d7-12ed-4a98-849c-dddbe1faf979 admin admin] PATH_INFO: `/v3/projects/4ba58ccb64164ca49cf7c7cc89fbc15d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:36:59.768855 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81f4d7-12ed-4a98-849c-dddbe1faf979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ba58ccb64164ca49cf7c7cc89fbc15d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:36:59.774807 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81f4d7-12ed-4a98-849c-dddbe1faf979 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:00.212807 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9f81f4d7-12ed-4a98-849c-dddbe1faf979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ba58ccb64164ca49cf7c7cc89fbc15d', 'request_id': 'req-9f81f4d7-12ed-4a98-849c-dddbe1faf979'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:00.222356 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9f81f4d7-12ed-4a98-849c-dddbe1faf979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ba58ccb64164ca49cf7c7cc89fbc15d', 'request_id': 'req-9f81f4d7-12ed-4a98-849c-dddbe1faf979'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:00.248430 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9f81f4d7-12ed-4a98-849c-dddbe1faf979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ba58ccb64164ca49cf7c7cc89fbc15d', 'request_id': 'req-9f81f4d7-12ed-4a98-849c-dddbe1faf979'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:00.393532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9f81f4d7-12ed-4a98-849c-dddbe1faf979 admin admin] The token cache is being invalidate because project 4ba58ccb64164ca49cf7c7cc89fbc15d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:00.393532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9f81f4d7-12ed-4a98-849c-dddbe1faf979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:00.403191 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9f81f4d7-12ed-4a98-849c-dddbe1faf979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:00.403191 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 927/1834] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:36:59 2022] DELETE /identity/v3/projects/4ba58ccb64164ca49cf7c7cc89fbc15d => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:00.404355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81f4d7-12ed-4a98-849c-dddbe1faf979 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:00.405328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9f81f4d7-12ed-4a98-849c-dddbe1faf979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:00.407272 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba0727-02a0-4d7e-a45b-59e498f236ca service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:00.412636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-05ba0727-02a0-4d7e-a45b-59e498f236ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:00.457758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba0727-02a0-4d7e-a45b-59e498f236ca service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:00.514883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81f4d7-12ed-4a98-849c-dddbe1faf979 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:00.520872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86240702-7fee-41e6-a29b-302836661270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:00.521686 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86240702-7fee-41e6-a29b-302836661270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:00.522038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86240702-7fee-41e6-a29b-302836661270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:00.522325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86240702-7fee-41e6-a29b-302836661270 admin admin] PATH_INFO: `/v3/users/6ca1a1d2c19b40828b4416c740db021f` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:00.523691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86240702-7fee-41e6-a29b-302836661270 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ca1a1d2c19b40828b4416c740db021f)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:00.526984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86240702-7fee-41e6-a29b-302836661270 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:00.634908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00663df2-26d2-4f35-afc7-35339ae8c204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:00.637848 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00663df2-26d2-4f35-afc7-35339ae8c204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:00.639947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00663df2-26d2-4f35-afc7-35339ae8c204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:00.640256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00663df2-26d2-4f35-afc7-35339ae8c204 admin admin] PATH_INFO: `/v3/projects/37da538b9ed248e3aabb57e3e008794a` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:00.642539 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00663df2-26d2-4f35-afc7-35339ae8c204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37da538b9ed248e3aabb57e3e008794a)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:00.645537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00663df2-26d2-4f35-afc7-35339ae8c204 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:01.016934 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-86240702-7fee-41e6-a29b-302836661270 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ca1a1d2c19b40828b4416c740db021f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:01.016934 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-86240702-7fee-41e6-a29b-302836661270 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ca1a1d2c19b40828b4416c740db021f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:01.040378 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-00663df2-26d2-4f35-afc7-35339ae8c204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37da538b9ed248e3aabb57e3e008794a', 'request_id': 'req-00663df2-26d2-4f35-afc7-35339ae8c204'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:01.054048 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-00663df2-26d2-4f35-afc7-35339ae8c204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37da538b9ed248e3aabb57e3e008794a', 'request_id': 'req-00663df2-26d2-4f35-afc7-35339ae8c204'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:01.067918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-00663df2-26d2-4f35-afc7-35339ae8c204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37da538b9ed248e3aabb57e3e008794a', 'request_id': 'req-00663df2-26d2-4f35-afc7-35339ae8c204'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:01.174973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-86240702-7fee-41e6-a29b-302836661270 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ca1a1d2c19b40828b4416c740db021f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:01.174973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-86240702-7fee-41e6-a29b-302836661270 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ca1a1d2c19b40828b4416c740db021f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:01.195044 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-86240702-7fee-41e6-a29b-302836661270 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ca1a1d2c19b40828b4416c740db021f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:01.233574 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 928/1835] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:00 2022] DELETE /identity/v3/users/6ca1a1d2c19b40828b4416c740db021f => generated 0 bytes in 830 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:01.235619 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86240702-7fee-41e6-a29b-302836661270 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:01.238591 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-86240702-7fee-41e6-a29b-302836661270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:01.244592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-00663df2-26d2-4f35-afc7-35339ae8c204 admin admin] The token cache is being invalidate because project 37da538b9ed248e3aabb57e3e008794a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:01.244592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-00663df2-26d2-4f35-afc7-35339ae8c204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:01.244592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-00663df2-26d2-4f35-afc7-35339ae8c204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:01.255371 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 908/1836] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:00 2022] DELETE /identity/v3/projects/37da538b9ed248e3aabb57e3e008794a => generated 0 bytes in 850 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:01.257505 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586283e6-5e4a-488d-90e4-e9bea7cd88a4 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:01.257791 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586283e6-5e4a-488d-90e4-e9bea7cd88a4 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:01.258076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586283e6-5e4a-488d-90e4-e9bea7cd88a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:01.295653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-586283e6-5e4a-488d-90e4-e9bea7cd88a4 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:01.298233 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-586283e6-5e4a-488d-90e4-e9bea7cd88a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:01.375077 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86240702-7fee-41e6-a29b-302836661270 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:01.382598 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3176e3-8ad5-4141-8f19-5107308bc3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:01.383324 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3176e3-8ad5-4141-8f19-5107308bc3ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:01.383604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3176e3-8ad5-4141-8f19-5107308bc3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:01.383877 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3176e3-8ad5-4141-8f19-5107308bc3ec admin admin] PATH_INFO: `/v3/projects/e462cb6b51f04221a55a34cd8ecece46` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:01.385339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3176e3-8ad5-4141-8f19-5107308bc3ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=e462cb6b51f04221a55a34cd8ecece46)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:01.388443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3176e3-8ad5-4141-8f19-5107308bc3ec admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:01.440252 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 909/1837] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:01.449548 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d557a-18eb-4431-b2d8-562ebbfa807f None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:01.450146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d557a-18eb-4431-b2d8-562ebbfa807f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:01.450668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d557a-18eb-4431-b2d8-562ebbfa807f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:01.481244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-908d557a-18eb-4431-b2d8-562ebbfa807f None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:01.481244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-908d557a-18eb-4431-b2d8-562ebbfa807f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:01.487543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 910/1838] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:01.491725 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d557a-18eb-4431-b2d8-562ebbfa807f None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:01.501201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d557a-18eb-4431-b2d8-562ebbfa807f None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:01.508754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5b9c7-bd1d-4385-af3a-c2d65dd8d155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:01.510234 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5b9c7-bd1d-4385-af3a-c2d65dd8d155 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:01.511516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5b9c7-bd1d-4385-af3a-c2d65dd8d155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:01.517487 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5b9c7-bd1d-4385-af3a-c2d65dd8d155 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:01.518582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f5b9c7-bd1d-4385-af3a-c2d65dd8d155 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:01.519094 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f5b9c7-bd1d-4385-af3a-c2d65dd8d155 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:01.521682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f5b9c7-bd1d-4385-af3a-c2d65dd8d155 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:01.537567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 911/1839] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:37:01.541688 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5b9c7-bd1d-4385-af3a-c2d65dd8d155 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:01.553175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5b9c7-bd1d-4385-af3a-c2d65dd8d155 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:01.558199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aab36b-9f7d-4528-8aa7-661d32cdf5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:01.559102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aab36b-9f7d-4528-8aa7-661d32cdf5c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:01.559479 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aab36b-9f7d-4528-8aa7-661d32cdf5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:01.559849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aab36b-9f7d-4528-8aa7-661d32cdf5c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:01.560586 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aab36b-9f7d-4528-8aa7-661d32cdf5c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:01.562961 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aab36b-9f7d-4528-8aa7-661d32cdf5c5 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:01.639356 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7f3176e3-8ad5-4141-8f19-5107308bc3ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e462cb6b51f04221a55a34cd8ecece46', 'request_id': 'req-7f3176e3-8ad5-4141-8f19-5107308bc3ec'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:01.646057 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7f3176e3-8ad5-4141-8f19-5107308bc3ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e462cb6b51f04221a55a34cd8ecece46', 'request_id': 'req-7f3176e3-8ad5-4141-8f19-5107308bc3ec'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:01.655801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7f3176e3-8ad5-4141-8f19-5107308bc3ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e462cb6b51f04221a55a34cd8ecece46', 'request_id': 'req-7f3176e3-8ad5-4141-8f19-5107308bc3ec'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:01.692486 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:01.727400 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7f3176e3-8ad5-4141-8f19-5107308bc3ec admin admin] The token cache is being invalidate because project e462cb6b51f04221a55a34cd8ecece46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:01.727926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7f3176e3-8ad5-4141-8f19-5107308bc3ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:01.728909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7f3176e3-8ad5-4141-8f19-5107308bc3ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:01.730315 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 912/1840] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:01 2022] POST /identity/v3/projects => generated 367 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:01.730783 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 929/1841] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:01 2022] DELETE /identity/v3/projects/e462cb6b51f04221a55a34cd8ecece46 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:01.734136 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aab36b-9f7d-4528-8aa7-661d32cdf5c5 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:01.734967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3176e3-8ad5-4141-8f19-5107308bc3ec admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:01.736419 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7f3176e3-8ad5-4141-8f19-5107308bc3ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:01.736536 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-32aab36b-9f7d-4528-8aa7-661d32cdf5c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:01.819472 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aab36b-9f7d-4528-8aa7-661d32cdf5c5 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:01.832922 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5b196-1177-4fb9-b4f1-f9c8bce476ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:01.833708 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5b196-1177-4fb9-b4f1-f9c8bce476ad admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:01.834085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5b196-1177-4fb9-b4f1-f9c8bce476ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:01.834200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3176e3-8ad5-4141-8f19-5107308bc3ec admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:01.834381 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5b196-1177-4fb9-b4f1-f9c8bce476ad admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:01.835590 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5b196-1177-4fb9-b4f1-f9c8bce476ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:01.838184 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5b196-1177-4fb9-b4f1-f9c8bce476ad admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:01.839434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de1011-571a-418f-8b8e-f71cc7b15e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:01.840045 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de1011-571a-418f-8b8e-f71cc7b15e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:01.840331 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de1011-571a-418f-8b8e-f71cc7b15e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:01.840613 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de1011-571a-418f-8b8e-f71cc7b15e70 admin admin] PATH_INFO: `/v3/users/b014e1df09434b728e6b76f8fcd38f16` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:01.841962 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de1011-571a-418f-8b8e-f71cc7b15e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b014e1df09434b728e6b76f8fcd38f16)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:01.844225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de1011-571a-418f-8b8e-f71cc7b15e70 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:01.927089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 913/1842] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:01 2022] POST /identity/v3/users => generated 284 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:01.930791 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5b196-1177-4fb9-b4f1-f9c8bce476ad admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:01.941274 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5b196-1177-4fb9-b4f1-f9c8bce476ad admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:01.948990 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f8e37-9e38-4eb7-8621-d47e59020871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:01.949610 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f8e37-9e38-4eb7-8621-d47e59020871 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:01.949891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f8e37-9e38-4eb7-8621-d47e59020871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:01.950168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f8e37-9e38-4eb7-8621-d47e59020871 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:01.950814 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f8e37-9e38-4eb7-8621-d47e59020871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:01.952418 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f8e37-9e38-4eb7-8621-d47e59020871 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:01.962866 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 914/1843] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:01.966588 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f8e37-9e38-4eb7-8621-d47e59020871 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:01.976592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f8e37-9e38-4eb7-8621-d47e59020871 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:01.981376 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836112a6-d6bb-4560-9287-211d10b50e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:01.982072 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836112a6-d6bb-4560-9287-211d10b50e81 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:01.982461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836112a6-d6bb-4560-9287-211d10b50e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:01.982785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836112a6-d6bb-4560-9287-211d10b50e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:01.983642 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836112a6-d6bb-4560-9287-211d10b50e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:01.985047 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836112a6-d6bb-4560-9287-211d10b50e81 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:01.996623 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 915/1844] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:02.000812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836112a6-d6bb-4560-9287-211d10b50e81 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:02.011587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836112a6-d6bb-4560-9287-211d10b50e81 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:02.016746 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a570cb-9fea-48ee-af5b-1ea273ded819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:02.017428 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a570cb-9fea-48ee-af5b-1ea273ded819 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:02.017711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a570cb-9fea-48ee-af5b-1ea273ded819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:02.018015 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a570cb-9fea-48ee-af5b-1ea273ded819 admin admin] PATH_INFO: `/v3/projects/702aec329384434b8a9494b50d113ad6/users/afa1974c791c48ad8cf23aeb46eb7dad/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:02.033040 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a570cb-9fea-48ee-af5b-1ea273ded819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=702aec329384434b8a9494b50d113ad6, user_id=afa1974c791c48ad8cf23aeb46eb7dad, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:02.035894 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a570cb-9fea-48ee-af5b-1ea273ded819 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:02.087032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-52de1011-571a-418f-8b8e-f71cc7b15e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b014e1df09434b728e6b76f8fcd38f16'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:02.087690 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-52de1011-571a-418f-8b8e-f71cc7b15e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b014e1df09434b728e6b76f8fcd38f16'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:02.155245 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-52de1011-571a-418f-8b8e-f71cc7b15e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b014e1df09434b728e6b76f8fcd38f16'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:02.158188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-52de1011-571a-418f-8b8e-f71cc7b15e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b014e1df09434b728e6b76f8fcd38f16'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:02.174545 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 916/1845] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:01 2022] PUT /identity/v3/projects/702aec329384434b8a9494b50d113ad6/users/afa1974c791c48ad8cf23aeb46eb7dad/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:02.174545 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a570cb-9fea-48ee-af5b-1ea273ded819 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:02.179030 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c3a570cb-9fea-48ee-af5b-1ea273ded819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:02.195115 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-52de1011-571a-418f-8b8e-f71cc7b15e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b014e1df09434b728e6b76f8fcd38f16'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:02.220352 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 930/1846] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:01 2022] DELETE /identity/v3/users/b014e1df09434b728e6b76f8fcd38f16 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:02.221801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07079bf3-e69a-4c09-90e8-548ad316d61c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:02.222207 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07079bf3-e69a-4c09-90e8-548ad316d61c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:02.222701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07079bf3-e69a-4c09-90e8-548ad316d61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:02.259903 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-07079bf3-e69a-4c09-90e8-548ad316d61c None None] MFA Rules not processed for user `afa1974c791c48ad8cf23aeb46eb7dad`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:02.262080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-07079bf3-e69a-4c09-90e8-548ad316d61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:02.274173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a570cb-9fea-48ee-af5b-1ea273ded819 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:02.341003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 931/1847] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:02.343559 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43712616-3687-40aa-846b-d752eb4c5873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:02.344234 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43712616-3687-40aa-846b-d752eb4c5873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:02.344684 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43712616-3687-40aa-846b-d752eb4c5873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:02.344896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43712616-3687-40aa-846b-d752eb4c5873 admin admin] PATH_INFO: `/v3/projects/6f84f2beacce4ed484a8e67c2d4c993e` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:02.345906 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07079bf3-e69a-4c09-90e8-548ad316d61c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:02.346700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43712616-3687-40aa-846b-d752eb4c5873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f84f2beacce4ed484a8e67c2d4c993e)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:02.347340 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-07079bf3-e69a-4c09-90e8-548ad316d61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:02.348630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43712616-3687-40aa-846b-d752eb4c5873 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:02.363790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07079bf3-e69a-4c09-90e8-548ad316d61c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:02.371021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9eca8-55c1-4b24-bee2-9156047fa36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:02.371233 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9eca8-55c1-4b24-bee2-9156047fa36b admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:02.371566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9eca8-55c1-4b24-bee2-9156047fa36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:02.371855 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9eca8-55c1-4b24-bee2-9156047fa36b admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:02.372497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9eca8-55c1-4b24-bee2-9156047fa36b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:02.375117 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9eca8-55c1-4b24-bee2-9156047fa36b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:02.520803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-43712616-3687-40aa-846b-d752eb4c5873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f84f2beacce4ed484a8e67c2d4c993e', 'request_id': 'req-43712616-3687-40aa-846b-d752eb4c5873'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:02.527319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 932/1848] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:02 2022] POST /identity/v3/projects => generated 367 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:02.528613 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-43712616-3687-40aa-846b-d752eb4c5873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f84f2beacce4ed484a8e67c2d4c993e', 'request_id': 'req-43712616-3687-40aa-846b-d752eb4c5873'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:02.531646 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9eca8-55c1-4b24-bee2-9156047fa36b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:02.536014 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-43712616-3687-40aa-846b-d752eb4c5873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f84f2beacce4ed484a8e67c2d4c993e', 'request_id': 'req-43712616-3687-40aa-846b-d752eb4c5873'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:02.604038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9eca8-55c1-4b24-bee2-9156047fa36b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:02.610581 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da24a1-58b3-4d2b-8b8d-9010fac92075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:02.611197 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da24a1-58b3-4d2b-8b8d-9010fac92075 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:02.611482 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da24a1-58b3-4d2b-8b8d-9010fac92075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:02.611755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da24a1-58b3-4d2b-8b8d-9010fac92075 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:02.612404 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da24a1-58b3-4d2b-8b8d-9010fac92075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:02.614361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da24a1-58b3-4d2b-8b8d-9010fac92075 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:02.676389 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-43712616-3687-40aa-846b-d752eb4c5873 admin admin] The token cache is being invalidate because project 6f84f2beacce4ed484a8e67c2d4c993e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:02.677044 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-43712616-3687-40aa-846b-d752eb4c5873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:02.677489 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-43712616-3687-40aa-846b-d752eb4c5873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:02.679621 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 917/1849] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:02 2022] DELETE /identity/v3/projects/6f84f2beacce4ed484a8e67c2d4c993e => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:02.684416 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e036d-36bf-45f4-8e55-cd1fb97f9535 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:02.684771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e036d-36bf-45f4-8e55-cd1fb97f9535 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:02.685185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e036d-36bf-45f4-8e55-cd1fb97f9535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:02.706685 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-5f7e036d-36bf-45f4-8e55-cd1fb97f9535 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:02.708181 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5f7e036d-36bf-45f4-8e55-cd1fb97f9535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:02.717518 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 918/1850] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:02.725114 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65b180-2003-4a38-b145-335740ea1102 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:02.725401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65b180-2003-4a38-b145-335740ea1102 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:02.725677 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65b180-2003-4a38-b145-335740ea1102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:02.735165 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 933/1851] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:02 2022] POST /identity/v3/users => generated 290 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:02.739909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da24a1-58b3-4d2b-8b8d-9010fac92075 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:02.741610 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-62da24a1-58b3-4d2b-8b8d-9010fac92075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:02.746383 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-de65b180-2003-4a38-b145-335740ea1102 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:02.747557 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-de65b180-2003-4a38-b145-335740ea1102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:02.753372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da24a1-58b3-4d2b-8b8d-9010fac92075 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:02.755481 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 919/1852] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:02.760656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65b180-2003-4a38-b145-335740ea1102 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:02.770348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65b180-2003-4a38-b145-335740ea1102 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:02.773662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefa732-3f5f-4846-b67d-40e8ae78965d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:02.774174 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefa732-3f5f-4846-b67d-40e8ae78965d admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:02.774550 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefa732-3f5f-4846-b67d-40e8ae78965d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:02.774711 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefa732-3f5f-4846-b67d-40e8ae78965d admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:02.775366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efefa732-3f5f-4846-b67d-40e8ae78965d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:02.775945 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc80d99-3811-46b1-abe7-6c6aafc0d5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:02.777345 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc80d99-3811-46b1-abe7-6c6aafc0d5ba admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:02.778089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc80d99-3811-46b1-abe7-6c6aafc0d5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:02.778089 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc80d99-3811-46b1-abe7-6c6aafc0d5ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:02.778875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc80d99-3811-46b1-abe7-6c6aafc0d5ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:02.779440 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc80d99-3811-46b1-abe7-6c6aafc0d5ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:02.779807 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efefa732-3f5f-4846-b67d-40e8ae78965d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:02.782372 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc80d99-3811-46b1-abe7-6c6aafc0d5ba admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:02.794507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 920/1853] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:37:02.796660 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 934/1854] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:02.798461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc80d99-3811-46b1-abe7-6c6aafc0d5ba admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:02.798638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefa732-3f5f-4846-b67d-40e8ae78965d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:02.813868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc80d99-3811-46b1-abe7-6c6aafc0d5ba admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:02.826401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefa732-3f5f-4846-b67d-40e8ae78965d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:02.827239 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4055d-ae33-48b4-8473-603fc8ea5849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:02.827923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4055d-ae33-48b4-8473-603fc8ea5849 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:02.828266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4055d-ae33-48b4-8473-603fc8ea5849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:02.828580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4055d-ae33-48b4-8473-603fc8ea5849 admin admin] PATH_INFO: `/v3/projects/9f78d50ed7674387a07f0fbe5e432bac/users/fcc5172457d54a3187f755416a0f7435/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:02.836731 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8c7c6-c08e-4e61-a9ef-a83e98bd16e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:02.837641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8c7c6-c08e-4e61-a9ef-a83e98bd16e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:02.837926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8c7c6-c08e-4e61-a9ef-a83e98bd16e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:02.838203 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8c7c6-c08e-4e61-a9ef-a83e98bd16e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:02.840685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8c7c6-c08e-4e61-a9ef-a83e98bd16e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:02.842669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8c7c6-c08e-4e61-a9ef-a83e98bd16e8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:02.864690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4055d-ae33-48b4-8473-603fc8ea5849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f78d50ed7674387a07f0fbe5e432bac, user_id=fcc5172457d54a3187f755416a0f7435, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:02.867720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4055d-ae33-48b4-8473-603fc8ea5849 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:02.944418 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 935/1855] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:02 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:02.945307 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8c7c6-c08e-4e61-a9ef-a83e98bd16e8 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:02.947108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f0f8c7c6-c08e-4e61-a9ef-a83e98bd16e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:02.972293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 921/1856] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:02 2022] PUT /identity/v3/projects/9f78d50ed7674387a07f0fbe5e432bac/users/fcc5172457d54a3187f755416a0f7435/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:02.975951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4055d-ae33-48b4-8473-603fc8ea5849 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:03.010682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8c7c6-c08e-4e61-a9ef-a83e98bd16e8 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:03.035824 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4055d-ae33-48b4-8473-603fc8ea5849 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:03.040469 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bdf65e-0df7-4a42-8f56-e04dfcdc047c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:03.041167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdf65e-0df7-4a42-8f56-e04dfcdc047c admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:03.041448 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdf65e-0df7-4a42-8f56-e04dfcdc047c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:03.042227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdf65e-0df7-4a42-8f56-e04dfcdc047c admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:03.042866 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bdf65e-0df7-4a42-8f56-e04dfcdc047c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:03.044820 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bdf65e-0df7-4a42-8f56-e04dfcdc047c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:03.048524 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:03.072373 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210126a-387a-46c7-9b85-7a5f20bd3be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:03.073053 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210126a-387a-46c7-9b85-7a5f20bd3be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:03.073385 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210126a-387a-46c7-9b85-7a5f20bd3be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:03.073665 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210126a-387a-46c7-9b85-7a5f20bd3be0 admin admin] PATH_INFO: `/v3/projects/8e1f2d4b68af47598f61ce902b3d8680` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:03.074799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210126a-387a-46c7-9b85-7a5f20bd3be0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e1f2d4b68af47598f61ce902b3d8680)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:03.078600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210126a-387a-46c7-9b85-7a5f20bd3be0 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:03.160522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 922/1857] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:02 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:03.161565 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bdf65e-0df7-4a42-8f56-e04dfcdc047c admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:03.171821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bdf65e-0df7-4a42-8f56-e04dfcdc047c admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:03.176544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d5c78-3ebd-4c58-a9db-8ec560b92909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:03.177166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d5c78-3ebd-4c58-a9db-8ec560b92909 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:03.177448 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d5c78-3ebd-4c58-a9db-8ec560b92909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:03.177723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d5c78-3ebd-4c58-a9db-8ec560b92909 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:03.178360 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d5c78-3ebd-4c58-a9db-8ec560b92909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:03.179962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d5c78-3ebd-4c58-a9db-8ec560b92909 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:03.200632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 923/1858] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:03.200632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d5c78-3ebd-4c58-a9db-8ec560b92909 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:03.202522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e210126a-387a-46c7-9b85-7a5f20bd3be0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e1f2d4b68af47598f61ce902b3d8680', 'request_id': 'req-e210126a-387a-46c7-9b85-7a5f20bd3be0'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:03.207257 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d5c78-3ebd-4c58-a9db-8ec560b92909 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:03.208941 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e210126a-387a-46c7-9b85-7a5f20bd3be0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e1f2d4b68af47598f61ce902b3d8680', 'request_id': 'req-e210126a-387a-46c7-9b85-7a5f20bd3be0'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:03.216140 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e210126a-387a-46c7-9b85-7a5f20bd3be0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e1f2d4b68af47598f61ce902b3d8680', 'request_id': 'req-e210126a-387a-46c7-9b85-7a5f20bd3be0'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:03.263524 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591907d9-adb8-4748-8252-522f81d7a539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:03.264052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591907d9-adb8-4748-8252-522f81d7a539 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:03.264374 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591907d9-adb8-4748-8252-522f81d7a539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:03.264655 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591907d9-adb8-4748-8252-522f81d7a539 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:03.265319 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591907d9-adb8-4748-8252-522f81d7a539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:03.267178 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591907d9-adb8-4748-8252-522f81d7a539 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:03.277227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 924/1859] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:03.277994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591907d9-adb8-4748-8252-522f81d7a539 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:03.287256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591907d9-adb8-4748-8252-522f81d7a539 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:03.292344 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c3c88-3097-4460-b8b3-004d0981526a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:03.293072 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c3c88-3097-4460-b8b3-004d0981526a admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:03.293379 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c3c88-3097-4460-b8b3-004d0981526a admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:03.293657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c3c88-3097-4460-b8b3-004d0981526a admin admin] PATH_INFO: `/v3/projects/9f78d50ed7674387a07f0fbe5e432bac/users/fcc5172457d54a3187f755416a0f7435/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:03.295543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c3c88-3097-4460-b8b3-004d0981526a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f78d50ed7674387a07f0fbe5e432bac, user_id=fcc5172457d54a3187f755416a0f7435, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:03.298182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c3c88-3097-4460-b8b3-004d0981526a admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:03.299631 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e210126a-387a-46c7-9b85-7a5f20bd3be0 admin admin] The token cache is being invalidate because project 8e1f2d4b68af47598f61ce902b3d8680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:03.299929 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e210126a-387a-46c7-9b85-7a5f20bd3be0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:03.300580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e210126a-387a-46c7-9b85-7a5f20bd3be0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:03.302163 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 936/1860] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:02 2022] DELETE /identity/v3/projects/8e1f2d4b68af47598f61ce902b3d8680 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:03.305790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210126a-387a-46c7-9b85-7a5f20bd3be0 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:03.305790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e210126a-387a-46c7-9b85-7a5f20bd3be0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:03.309395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 925/1861] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:03 2022] PUT /identity/v3/projects/9f78d50ed7674387a07f0fbe5e432bac/users/fcc5172457d54a3187f755416a0f7435/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:03.310652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846c7ec-78ad-41a6-8666-198028066664 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:03.310933 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846c7ec-78ad-41a6-8666-198028066664 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:03.311206 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846c7ec-78ad-41a6-8666-198028066664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:03.317372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210126a-387a-46c7-9b85-7a5f20bd3be0 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:03.330132 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-8846c7ec-78ad-41a6-8666-198028066664 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:03.332160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-8846c7ec-78ad-41a6-8666-198028066664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:03.374811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7468e4-3b20-45f6-a5db-266c063a3dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:03.375432 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7468e4-3b20-45f6-a5db-266c063a3dca admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:03.375720 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7468e4-3b20-45f6-a5db-266c063a3dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:03.375994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7468e4-3b20-45f6-a5db-266c063a3dca admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:03.376672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7468e4-3b20-45f6-a5db-266c063a3dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:03.378351 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7468e4-3b20-45f6-a5db-266c063a3dca admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:03.388138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 937/1862] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:03.389545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e0044-0936-4b43-b22f-f7e8ff088bcf None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:03.389662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e0044-0936-4b43-b22f-f7e8ff088bcf None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:03.389910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e0044-0936-4b43-b22f-f7e8ff088bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:03.391316 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 926/1863] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:03.400693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569a48b-8acb-46ab-a27c-c71d67e16dc7 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:03.401678 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569a48b-8acb-46ab-a27c-c71d67e16dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:03.402034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569a48b-8acb-46ab-a27c-c71d67e16dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:03.418633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-210e0044-0936-4b43-b22f-f7e8ff088bcf None None] MFA Rules not processed for user `fcc5172457d54a3187f755416a0f7435`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:03.419861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-210e0044-0936-4b43-b22f-f7e8ff088bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:03.422693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-e569a48b-8acb-46ab-a27c-c71d67e16dc7 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:03.423928 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e569a48b-8acb-46ab-a27c-c71d67e16dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:03.434173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 927/1864] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:03.434985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569a48b-8acb-46ab-a27c-c71d67e16dc7 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:03.445703 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569a48b-8acb-46ab-a27c-c71d67e16dc7 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:03.450914 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd111e-a41c-41b0-8c22-417da6c3ffbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:03.451523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd111e-a41c-41b0-8c22-417da6c3ffbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:03.451771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd111e-a41c-41b0-8c22-417da6c3ffbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:03.452045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd111e-a41c-41b0-8c22-417da6c3ffbe admin admin] PATH_INFO: `/v3/projects/42f07e17c66a439e878a6ee0b1d93314/users/276c75cfbbeb4a17b27004fade720adb/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:03.492827 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd111e-a41c-41b0-8c22-417da6c3ffbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f07e17c66a439e878a6ee0b1d93314, user_id=276c75cfbbeb4a17b27004fade720adb, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:03.500581 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd111e-a41c-41b0-8c22-417da6c3ffbe admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:03.522079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 938/1865] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:03 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:03.523000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e0044-0936-4b43-b22f-f7e8ff088bcf None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:03.539458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e0044-0936-4b43-b22f-f7e8ff088bcf None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:03.545699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c69e67-81f7-4b4e-916c-ac786e939e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:03.546330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c69e67-81f7-4b4e-916c-ac786e939e0a admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:03.546612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c69e67-81f7-4b4e-916c-ac786e939e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:03.546890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c69e67-81f7-4b4e-916c-ac786e939e0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:03.547826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c69e67-81f7-4b4e-916c-ac786e939e0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:03.548143 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c69e67-81f7-4b4e-916c-ac786e939e0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:03.551230 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c69e67-81f7-4b4e-916c-ac786e939e0a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:03.567542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 939/1866] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:37:03.568435 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c69e67-81f7-4b4e-916c-ac786e939e0a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:03.574864 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-68c69e67-81f7-4b4e-916c-ac786e939e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:03.605311 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 928/1867] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:03 2022] PUT /identity/v3/projects/42f07e17c66a439e878a6ee0b1d93314/users/276c75cfbbeb4a17b27004fade720adb/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:03.606930 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd111e-a41c-41b0-8c22-417da6c3ffbe admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:03.671999 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c69e67-81f7-4b4e-916c-ac786e939e0a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:03.747275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd111e-a41c-41b0-8c22-417da6c3ffbe admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:03.752088 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59902279-c1db-470c-967d-07b1d2e94cba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:03.753128 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59902279-c1db-470c-967d-07b1d2e94cba service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:03.753519 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59902279-c1db-470c-967d-07b1d2e94cba service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:03.753914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59902279-c1db-470c-967d-07b1d2e94cba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:03.764140 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38bd5e-7bcb-4b42-be30-b71c545378e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:03.764990 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38bd5e-7bcb-4b42-be30-b71c545378e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:03.765313 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38bd5e-7bcb-4b42-be30-b71c545378e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:03.766872 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38bd5e-7bcb-4b42-be30-b71c545378e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:03.767200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38bd5e-7bcb-4b42-be30-b71c545378e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:03.773201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38bd5e-7bcb-4b42-be30-b71c545378e8 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:03.863477 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59902279-c1db-470c-967d-07b1d2e94cba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:03.864986 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59902279-c1db-470c-967d-07b1d2e94cba service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:03.878797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 940/1868] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:03 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:03.881791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464c2bc-bd2a-4aa3-a8df-a7d5b14acd11 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:03.882055 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464c2bc-bd2a-4aa3-a8df-a7d5b14acd11 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:03.882329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464c2bc-bd2a-4aa3-a8df-a7d5b14acd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:03.929538 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 929/1869] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:03 2022] POST /identity/v3/projects => generated 367 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:03.933205 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38bd5e-7bcb-4b42-be30-b71c545378e8 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:03.933916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a464c2bc-bd2a-4aa3-a8df-a7d5b14acd11 None None] MFA Rules not processed for user `276c75cfbbeb4a17b27004fade720adb`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:03.935391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a464c2bc-bd2a-4aa3-a8df-a7d5b14acd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:04.020880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 941/1870] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:04.021795 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464c2bc-bd2a-4aa3-a8df-a7d5b14acd11 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:04.028132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a464c2bc-bd2a-4aa3-a8df-a7d5b14acd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:04.077147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38bd5e-7bcb-4b42-be30-b71c545378e8 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:04.085050 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dca0ce-fd67-441e-a83f-bc14fa7dd2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:04.085050 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dca0ce-fd67-441e-a83f-bc14fa7dd2d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:04.085050 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dca0ce-fd67-441e-a83f-bc14fa7dd2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:04.085050 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dca0ce-fd67-441e-a83f-bc14fa7dd2d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:04.085478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dca0ce-fd67-441e-a83f-bc14fa7dd2d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:04.086571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dca0ce-fd67-441e-a83f-bc14fa7dd2d6 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:04.091875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:04.148033 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464c2bc-bd2a-4aa3-a8df-a7d5b14acd11 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:04.154234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90507ce-f5e1-4f79-9dfb-da687a78bb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:04.155970 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90507ce-f5e1-4f79-9dfb-da687a78bb9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:04.155970 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90507ce-f5e1-4f79-9dfb-da687a78bb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:04.155970 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90507ce-f5e1-4f79-9dfb-da687a78bb9b admin admin] PATH_INFO: `/v3/users/5c1fa0a4988c4934a281a2d00880c925` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:04.156593 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90507ce-f5e1-4f79-9dfb-da687a78bb9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c1fa0a4988c4934a281a2d00880c925)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:04.158701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90507ce-f5e1-4f79-9dfb-da687a78bb9b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:04.189443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 930/1871] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:03 2022] POST /identity/v3/users => generated 284 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:37:04.190782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dca0ce-fd67-441e-a83f-bc14fa7dd2d6 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:04.205519 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dca0ce-fd67-441e-a83f-bc14fa7dd2d6 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:04.211150 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165eee3b-7bf2-4210-a62b-033a3fe94ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:04.214245 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165eee3b-7bf2-4210-a62b-033a3fe94ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:04.214245 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165eee3b-7bf2-4210-a62b-033a3fe94ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:04.214245 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165eee3b-7bf2-4210-a62b-033a3fe94ce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:04.214245 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165eee3b-7bf2-4210-a62b-033a3fe94ce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:04.215824 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165eee3b-7bf2-4210-a62b-033a3fe94ce9 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:04.316073 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 931/1872] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:04 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:04.316538 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165eee3b-7bf2-4210-a62b-033a3fe94ce9 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:04.363808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a90507ce-f5e1-4f79-9dfb-da687a78bb9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c1fa0a4988c4934a281a2d00880c925'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:04.369582 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a90507ce-f5e1-4f79-9dfb-da687a78bb9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c1fa0a4988c4934a281a2d00880c925'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:04.399813 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165eee3b-7bf2-4210-a62b-033a3fe94ce9 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:04.405303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ae480-8add-4291-9a14-c0e63fc7e769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:04.407761 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ae480-8add-4291-9a14-c0e63fc7e769 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:04.407761 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ae480-8add-4291-9a14-c0e63fc7e769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:04.407761 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ae480-8add-4291-9a14-c0e63fc7e769 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:04.407761 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ae480-8add-4291-9a14-c0e63fc7e769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:04.408809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ae480-8add-4291-9a14-c0e63fc7e769 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:04.421194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 932/1873] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:04.421194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ae480-8add-4291-9a14-c0e63fc7e769 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:04.448461 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a90507ce-f5e1-4f79-9dfb-da687a78bb9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c1fa0a4988c4934a281a2d00880c925'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:04.449128 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a90507ce-f5e1-4f79-9dfb-da687a78bb9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c1fa0a4988c4934a281a2d00880c925'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:04.462097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a90507ce-f5e1-4f79-9dfb-da687a78bb9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c1fa0a4988c4934a281a2d00880c925'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:04.474744 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 942/1874] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:04 2022] DELETE /identity/v3/users/5c1fa0a4988c4934a281a2d00880c925 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:04.475700 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90507ce-f5e1-4f79-9dfb-da687a78bb9b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:04.478882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a90507ce-f5e1-4f79-9dfb-da687a78bb9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:04.480942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ae480-8add-4291-9a14-c0e63fc7e769 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:04.530889 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cb14d-1008-4bb9-b849-695c98ff6718 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:04.531507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cb14d-1008-4bb9-b849-695c98ff6718 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:04.531779 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cb14d-1008-4bb9-b849-695c98ff6718 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:04.532050 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cb14d-1008-4bb9-b849-695c98ff6718 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:04.533798 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c11cb14d-1008-4bb9-b849-695c98ff6718 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:04.544511 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90507ce-f5e1-4f79-9dfb-da687a78bb9b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:04.549730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567f5ea-0179-4e9f-989c-0915addd07ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:04.550114 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567f5ea-0179-4e9f-989c-0915addd07ca admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:04.550445 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567f5ea-0179-4e9f-989c-0915addd07ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:04.550812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567f5ea-0179-4e9f-989c-0915addd07ca admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:04.551368 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b567f5ea-0179-4e9f-989c-0915addd07ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:04.553339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b567f5ea-0179-4e9f-989c-0915addd07ca admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:04.574434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:04.593497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11cb14d-1008-4bb9-b849-695c98ff6718 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:04.594938 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11cb14d-1008-4bb9-b849-695c98ff6718 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:04.603790 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 933/1875] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:04.605144 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cb14d-1008-4bb9-b849-695c98ff6718 service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:04.606647 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c11cb14d-1008-4bb9-b849-695c98ff6718 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:04.621254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cb14d-1008-4bb9-b849-695c98ff6718 service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:04.626094 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093fca1-70d6-41eb-b8fb-257657d3db2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:04.629419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093fca1-70d6-41eb-b8fb-257657d3db2d admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:04.629419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093fca1-70d6-41eb-b8fb-257657d3db2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:04.629419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093fca1-70d6-41eb-b8fb-257657d3db2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:04.629419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2093fca1-70d6-41eb-b8fb-257657d3db2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:04.629827 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2093fca1-70d6-41eb-b8fb-257657d3db2d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:04.639496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 934/1876] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:04.640608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9925e-0384-42a0-b2ac-d668909df0da None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:04.640935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9925e-0384-42a0-b2ac-d668909df0da None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:04.641487 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9925e-0384-42a0-b2ac-d668909df0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:04.659095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 943/1877] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:04 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:04.659095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567f5ea-0179-4e9f-989c-0915addd07ca admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:04.664377 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-2ae9925e-0384-42a0-b2ac-d668909df0da None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:04.665621 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-2ae9925e-0384-42a0-b2ac-d668909df0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:04.668059 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567f5ea-0179-4e9f-989c-0915addd07ca admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:04.673105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a46678-ea20-4291-a36e-c64d51c2eb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:04.673861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a46678-ea20-4291-a36e-c64d51c2eb9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:04.674213 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a46678-ea20-4291-a36e-c64d51c2eb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:04.674871 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 935/1878] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:04.675306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9925e-0384-42a0-b2ac-d668909df0da None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:04.676108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a46678-ea20-4291-a36e-c64d51c2eb9a admin admin] PATH_INFO: `/v3/projects/5346838c646e4c64bb9b6bb48b09fd71/users/7f4f6efe430a45198b1adeb9f6a5a527/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:04.693710 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9925e-0384-42a0-b2ac-d668909df0da None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:04.694893 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a46678-ea20-4291-a36e-c64d51c2eb9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5346838c646e4c64bb9b6bb48b09fd71, user_id=7f4f6efe430a45198b1adeb9f6a5a527, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:04.697618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a46678-ea20-4291-a36e-c64d51c2eb9a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:04.700546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29192d-06dd-4ac8-b504-7217bed0de21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:04.701216 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29192d-06dd-4ac8-b504-7217bed0de21 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:04.701616 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29192d-06dd-4ac8-b504-7217bed0de21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:04.701881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29192d-06dd-4ac8-b504-7217bed0de21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:04.702540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f29192d-06dd-4ac8-b504-7217bed0de21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:04.704155 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f29192d-06dd-4ac8-b504-7217bed0de21 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:04.715206 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 936/1879] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:04.715780 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29192d-06dd-4ac8-b504-7217bed0de21 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:04.717512 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-8f29192d-06dd-4ac8-b504-7217bed0de21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:04.777753 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29192d-06dd-4ac8-b504-7217bed0de21 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:04.781623 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63733048-5304-400a-a283-5d4a7428bd2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:04.782256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63733048-5304-400a-a283-5d4a7428bd2c service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:04.782544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63733048-5304-400a-a283-5d4a7428bd2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:04.782819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63733048-5304-400a-a283-5d4a7428bd2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:04.792593 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63733048-5304-400a-a283-5d4a7428bd2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:04.794046 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63733048-5304-400a-a283-5d4a7428bd2c service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:04.801530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 944/1880] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:04 2022] PUT /identity/v3/projects/5346838c646e4c64bb9b6bb48b09fd71/users/7f4f6efe430a45198b1adeb9f6a5a527/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:04.801530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a46678-ea20-4291-a36e-c64d51c2eb9a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:04.804165 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 937/1881] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:04.807819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87942d48-bc22-418f-8e0e-7534c7d9df36 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:04.808098 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87942d48-bc22-418f-8e0e-7534c7d9df36 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:04.808941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87942d48-bc22-418f-8e0e-7534c7d9df36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:04.849747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-87942d48-bc22-418f-8e0e-7534c7d9df36 None None] MFA Rules not processed for user `7f4f6efe430a45198b1adeb9f6a5a527`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:04.851072 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-87942d48-bc22-418f-8e0e-7534c7d9df36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:04.890399 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a46678-ea20-4291-a36e-c64d51c2eb9a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:04.895605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95534e-e437-40dc-9112-c30d7fbb5323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:04.896302 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95534e-e437-40dc-9112-c30d7fbb5323 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:04.896863 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95534e-e437-40dc-9112-c30d7fbb5323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:04.896863 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95534e-e437-40dc-9112-c30d7fbb5323 admin admin] PATH_INFO: `/v3/projects/e801bccdf7114dbc8856179365584a80/users/7b3011ebe40a42399fb6ce58fb00176c/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:04.914107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95534e-e437-40dc-9112-c30d7fbb5323 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e801bccdf7114dbc8856179365584a80, user_id=7b3011ebe40a42399fb6ce58fb00176c, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:04.916717 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95534e-e437-40dc-9112-c30d7fbb5323 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:04.924649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 938/1882] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:04.929189 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87942d48-bc22-418f-8e0e-7534c7d9df36 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:04.934800 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:04.940250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87942d48-bc22-418f-8e0e-7534c7d9df36 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:04.944986 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2a3e8-9976-4fff-a493-c96dbe589e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:04.945609 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2a3e8-9976-4fff-a493-c96dbe589e16 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:04.945891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2a3e8-9976-4fff-a493-c96dbe589e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:04.946168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2a3e8-9976-4fff-a493-c96dbe589e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:04.946796 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2a3e8-9976-4fff-a493-c96dbe589e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:04.949496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2a3e8-9976-4fff-a493-c96dbe589e16 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:05.009660 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 945/1883] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:04 2022] PUT /identity/v3/projects/e801bccdf7114dbc8856179365584a80/users/7b3011ebe40a42399fb6ce58fb00176c/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:05.014886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95534e-e437-40dc-9112-c30d7fbb5323 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:05.063806 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 939/1884] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:04 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:37:05.067413 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2a3e8-9976-4fff-a493-c96dbe589e16 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:05.103428 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95534e-e437-40dc-9112-c30d7fbb5323 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:05.150686 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2a3e8-9976-4fff-a493-c96dbe589e16 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:05.156786 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca36c9f-fa3c-4447-b787-9698d557af10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:05.157690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca36c9f-fa3c-4447-b787-9698d557af10 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:05.158149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca36c9f-fa3c-4447-b787-9698d557af10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:05.158529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca36c9f-fa3c-4447-b787-9698d557af10 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:05.159540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca36c9f-fa3c-4447-b787-9698d557af10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:05.161720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca36c9f-fa3c-4447-b787-9698d557af10 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:05.195710 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e2bfd-61c5-4990-9cec-1d8f03d50b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:05.199582 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e2bfd-61c5-4990-9cec-1d8f03d50b1f admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:05.199582 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e2bfd-61c5-4990-9cec-1d8f03d50b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:05.199582 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e2bfd-61c5-4990-9cec-1d8f03d50b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:05.199582 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e2bfd-61c5-4990-9cec-1d8f03d50b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:05.199981 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e2bfd-61c5-4990-9cec-1d8f03d50b1f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:05.210995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 946/1885] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:05.215787 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e2bfd-61c5-4990-9cec-1d8f03d50b1f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:05.232678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e2bfd-61c5-4990-9cec-1d8f03d50b1f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:05.238597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085d8df-5cea-44f1-973d-e9c11f35421b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:05.239348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085d8df-5cea-44f1-973d-e9c11f35421b admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:05.239699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085d8df-5cea-44f1-973d-e9c11f35421b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:05.240051 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085d8df-5cea-44f1-973d-e9c11f35421b admin admin] PATH_INFO: `/v3/projects/e801bccdf7114dbc8856179365584a80/users/7b3011ebe40a42399fb6ce58fb00176c/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:05.242831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085d8df-5cea-44f1-973d-e9c11f35421b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e801bccdf7114dbc8856179365584a80, user_id=7b3011ebe40a42399fb6ce58fb00176c, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:05.256460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085d8df-5cea-44f1-973d-e9c11f35421b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:05.265504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 947/1886] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:05 2022] PUT /identity/v3/projects/e801bccdf7114dbc8856179365584a80/users/7b3011ebe40a42399fb6ce58fb00176c/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:05.265504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085d8df-5cea-44f1-973d-e9c11f35421b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:05.271740 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b085d8df-5cea-44f1-973d-e9c11f35421b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:05.288334 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 940/1887] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:05 2022] POST /identity/v3/users => generated 291 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:37:05.289785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37363e63-7fc8-434e-8ed7-d47370424656 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:05.290076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37363e63-7fc8-434e-8ed7-d47370424656 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:05.290362 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37363e63-7fc8-434e-8ed7-d47370424656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:05.391525 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-37363e63-7fc8-434e-8ed7-d47370424656 None None] MFA Rules not processed for user `7b3011ebe40a42399fb6ce58fb00176c`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:05.392757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-37363e63-7fc8-434e-8ed7-d47370424656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:05.412461 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085d8df-5cea-44f1-973d-e9c11f35421b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:05.417548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43766c76-af08-4ae0-b68b-3cc2acdd2846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:05.418243 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43766c76-af08-4ae0-b68b-3cc2acdd2846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:05.418814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43766c76-af08-4ae0-b68b-3cc2acdd2846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:05.418814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43766c76-af08-4ae0-b68b-3cc2acdd2846 admin admin] PATH_INFO: `/v3/projects/88dc91cd104242938a2dec34b6188830` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:05.420073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43766c76-af08-4ae0-b68b-3cc2acdd2846 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88dc91cd104242938a2dec34b6188830)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:05.423571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43766c76-af08-4ae0-b68b-3cc2acdd2846 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:05.488726 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 941/1888] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:05.488726 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37363e63-7fc8-434e-8ed7-d47370424656 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:05.500783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37363e63-7fc8-434e-8ed7-d47370424656 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:05.505135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be1d06-c0e3-4a77-aa8c-cb2eea774b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:05.505976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be1d06-c0e3-4a77-aa8c-cb2eea774b6f admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:05.506350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be1d06-c0e3-4a77-aa8c-cb2eea774b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:05.506722 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be1d06-c0e3-4a77-aa8c-cb2eea774b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:05.507489 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be1d06-c0e3-4a77-aa8c-cb2eea774b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:05.509221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be1d06-c0e3-4a77-aa8c-cb2eea774b6f admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:05.528622 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 942/1889] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:05.530471 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be1d06-c0e3-4a77-aa8c-cb2eea774b6f admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:05.532335 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d4be1d06-c0e3-4a77-aa8c-cb2eea774b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:05.546487 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-43766c76-af08-4ae0-b68b-3cc2acdd2846 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88dc91cd104242938a2dec34b6188830', 'request_id': 'req-43766c76-af08-4ae0-b68b-3cc2acdd2846'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:05.564159 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-43766c76-af08-4ae0-b68b-3cc2acdd2846 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88dc91cd104242938a2dec34b6188830', 'request_id': 'req-43766c76-af08-4ae0-b68b-3cc2acdd2846'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:05.573916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-43766c76-af08-4ae0-b68b-3cc2acdd2846 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88dc91cd104242938a2dec34b6188830', 'request_id': 'req-43766c76-af08-4ae0-b68b-3cc2acdd2846'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:05.623362 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be1d06-c0e3-4a77-aa8c-cb2eea774b6f admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:05.672139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214de63e-df19-4523-9f19-2257a3db64d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:05.672741 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214de63e-df19-4523-9f19-2257a3db64d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:05.673019 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214de63e-df19-4523-9f19-2257a3db64d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:05.673324 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214de63e-df19-4523-9f19-2257a3db64d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:05.693710 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-43766c76-af08-4ae0-b68b-3cc2acdd2846 admin admin] The token cache is being invalidate because project 88dc91cd104242938a2dec34b6188830 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:05.698091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-43766c76-af08-4ae0-b68b-3cc2acdd2846 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:05.698091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-43766c76-af08-4ae0-b68b-3cc2acdd2846 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:05.698091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 948/1890] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:05 2022] DELETE /identity/v3/projects/88dc91cd104242938a2dec34b6188830 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:05.698091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43766c76-af08-4ae0-b68b-3cc2acdd2846 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:05.698404 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-43766c76-af08-4ae0-b68b-3cc2acdd2846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:05.733961 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214de63e-df19-4523-9f19-2257a3db64d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:05.735443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214de63e-df19-4523-9f19-2257a3db64d7 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:05.745086 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 943/1891] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:05.745893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214de63e-df19-4523-9f19-2257a3db64d7 service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:05.762153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43766c76-af08-4ae0-b68b-3cc2acdd2846 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:05.775647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ba744-1b41-4288-a7f9-d1652c92b425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:05.777012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ba744-1b41-4288-a7f9-d1652c92b425 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:05.777327 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ba744-1b41-4288-a7f9-d1652c92b425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:05.781490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ba744-1b41-4288-a7f9-d1652c92b425 admin admin] PATH_INFO: `/v3/projects/980bf4df217d4ccfad8d808d840d414e/users/bb554166f21446eb9655c277ee3edbc1/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:05.798411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ba744-1b41-4288-a7f9-d1652c92b425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=980bf4df217d4ccfad8d808d840d414e, user_id=bb554166f21446eb9655c277ee3edbc1, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:05.803722 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ba744-1b41-4288-a7f9-d1652c92b425 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:05.816005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214de63e-df19-4523-9f19-2257a3db64d7 service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:05.821819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa34bc-9ce1-42f1-b54f-33713e2bfcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:05.822464 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa34bc-9ce1-42f1-b54f-33713e2bfcb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:05.822747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa34bc-9ce1-42f1-b54f-33713e2bfcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:05.823678 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa34bc-9ce1-42f1-b54f-33713e2bfcb1 admin admin] PATH_INFO: `/v3/users/3168c1084c9f46ad8d19030bcd1f3bcc` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:05.824980 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa34bc-9ce1-42f1-b54f-33713e2bfcb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3168c1084c9f46ad8d19030bcd1f3bcc)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:05.827930 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa34bc-9ce1-42f1-b54f-33713e2bfcb1 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:05.858492 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:05.894790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 949/1892] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:05 2022] PUT /identity/v3/projects/980bf4df217d4ccfad8d808d840d414e/users/bb554166f21446eb9655c277ee3edbc1/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:05.895751 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ba744-1b41-4288-a7f9-d1652c92b425 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:05.897459 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-832ba744-1b41-4288-a7f9-d1652c92b425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:05.952878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ba744-1b41-4288-a7f9-d1652c92b425 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:05.956554 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68594a39-bbdc-4fba-9c85-63439b27dee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:05.957188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68594a39-bbdc-4fba-9c85-63439b27dee0 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:05.957468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68594a39-bbdc-4fba-9c85-63439b27dee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:05.957784 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68594a39-bbdc-4fba-9c85-63439b27dee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:05.959476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-68594a39-bbdc-4fba-9c85-63439b27dee0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:06.008305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-fbaa34bc-9ce1-42f1-b54f-33713e2bfcb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3168c1084c9f46ad8d19030bcd1f3bcc'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:06.021392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68594a39-bbdc-4fba-9c85-63439b27dee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:06.022817 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68594a39-bbdc-4fba-9c85-63439b27dee0 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:06.032262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 950/1893] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:06.033045 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68594a39-bbdc-4fba-9c85-63439b27dee0 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:06.080752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-fbaa34bc-9ce1-42f1-b54f-33713e2bfcb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3168c1084c9f46ad8d19030bcd1f3bcc'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:06.081403 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-fbaa34bc-9ce1-42f1-b54f-33713e2bfcb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3168c1084c9f46ad8d19030bcd1f3bcc'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:06.082329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-fbaa34bc-9ce1-42f1-b54f-33713e2bfcb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3168c1084c9f46ad8d19030bcd1f3bcc'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:06.092899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-fbaa34bc-9ce1-42f1-b54f-33713e2bfcb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3168c1084c9f46ad8d19030bcd1f3bcc'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:06.101818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68594a39-bbdc-4fba-9c85-63439b27dee0 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:06.106499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5841988-26b6-4a75-b5b4-b9a6fb6f56f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:06.108626 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5841988-26b6-4a75-b5b4-b9a6fb6f56f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:06.108941 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5841988-26b6-4a75-b5b4-b9a6fb6f56f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:06.109269 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5841988-26b6-4a75-b5b4-b9a6fb6f56f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:06.109917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5841988-26b6-4a75-b5b4-b9a6fb6f56f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:06.111513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5841988-26b6-4a75-b5b4-b9a6fb6f56f5 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:06.121390 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 944/1894] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:05 2022] DELETE /identity/v3/users/3168c1084c9f46ad8d19030bcd1f3bcc => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:06.122795 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 951/1895] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:06.128937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa34bc-9ce1-42f1-b54f-33713e2bfcb1 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:06.130372 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-fbaa34bc-9ce1-42f1-b54f-33713e2bfcb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:06.223038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa34bc-9ce1-42f1-b54f-33713e2bfcb1 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:06.228156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f1dcf-b571-4290-bc1b-0ce84a4d7c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:06.229724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f1dcf-b571-4290-bc1b-0ce84a4d7c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:06.230260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f1dcf-b571-4290-bc1b-0ce84a4d7c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:06.230913 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f1dcf-b571-4290-bc1b-0ce84a4d7c2f admin admin] PATH_INFO: `/v3/projects/980bf4df217d4ccfad8d808d840d414e/users/bb554166f21446eb9655c277ee3edbc1/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:06.233001 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f1dcf-b571-4290-bc1b-0ce84a4d7c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=980bf4df217d4ccfad8d808d840d414e, user_id=bb554166f21446eb9655c277ee3edbc1, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:06.237048 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f1dcf-b571-4290-bc1b-0ce84a4d7c2f admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:06.248986 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 945/1896] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:06 2022] PUT /identity/v3/projects/980bf4df217d4ccfad8d808d840d414e/users/bb554166f21446eb9655c277ee3edbc1/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:06.259460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73e9c4-7b70-417e-a7f6-b311d72de253 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:06.259744 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73e9c4-7b70-417e-a7f6-b311d72de253 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:06.260047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73e9c4-7b70-417e-a7f6-b311d72de253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:06.290461 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1e73e9c4-7b70-417e-a7f6-b311d72de253 None None] MFA Rules not processed for user `bb554166f21446eb9655c277ee3edbc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:06.292414 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1e73e9c4-7b70-417e-a7f6-b311d72de253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:06.361718 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 952/1897] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:06.373708 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f1dcf-b571-4290-bc1b-0ce84a4d7c2f admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:06.376312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-f61f1dcf-b571-4290-bc1b-0ce84a4d7c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:06.473413 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f1dcf-b571-4290-bc1b-0ce84a4d7c2f admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:06.478144 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a1241-4d5d-4c44-af73-8e00ce5aa353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:06.480456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a1241-4d5d-4c44-af73-8e00ce5aa353 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:06.481402 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a1241-4d5d-4c44-af73-8e00ce5aa353 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:06.482117 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a1241-4d5d-4c44-af73-8e00ce5aa353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:06.496474 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a1241-4d5d-4c44-af73-8e00ce5aa353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:06.498285 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a1241-4d5d-4c44-af73-8e00ce5aa353 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:06.536261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 946/1898] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:06.704318 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:DELETE=6,keystone:SELECT=88,keystone:INSERT=6 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:06.767018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73e9c4-7b70-417e-a7f6-b311d72de253 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:06.768676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1e73e9c4-7b70-417e-a7f6-b311d72de253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:06.915351 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73e9c4-7b70-417e-a7f6-b311d72de253 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:06.933728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d0002-344e-4429-9b70-3a28fbbdb855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:06.934960 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d0002-344e-4429-9b70-3a28fbbdb855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:06.935459 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d0002-344e-4429-9b70-3a28fbbdb855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:06.941911 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d0002-344e-4429-9b70-3a28fbbdb855 admin admin] PATH_INFO: `/v3/projects/480545e4a7954166b8afe6be2690e1b9` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:06.951413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d0002-344e-4429-9b70-3a28fbbdb855 admin admin] RBAC: Authorizing `identity:delete_project(project_id=480545e4a7954166b8afe6be2690e1b9)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:06.953418 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d0002-344e-4429-9b70-3a28fbbdb855 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:07.054400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:07.074683 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ed8d0002-344e-4429-9b70-3a28fbbdb855 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '480545e4a7954166b8afe6be2690e1b9', 'request_id': 'req-ed8d0002-344e-4429-9b70-3a28fbbdb855'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:07.081117 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ed8d0002-344e-4429-9b70-3a28fbbdb855 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '480545e4a7954166b8afe6be2690e1b9', 'request_id': 'req-ed8d0002-344e-4429-9b70-3a28fbbdb855'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:07.088120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ed8d0002-344e-4429-9b70-3a28fbbdb855 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '480545e4a7954166b8afe6be2690e1b9', 'request_id': 'req-ed8d0002-344e-4429-9b70-3a28fbbdb855'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:07.195050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ed8d0002-344e-4429-9b70-3a28fbbdb855 admin admin] The token cache is being invalidate because project 480545e4a7954166b8afe6be2690e1b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:07.197207 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ed8d0002-344e-4429-9b70-3a28fbbdb855 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:07.197207 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ed8d0002-344e-4429-9b70-3a28fbbdb855 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:07.198320 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 953/1899] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:06 2022] DELETE /identity/v3/projects/480545e4a7954166b8afe6be2690e1b9 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:07.203192 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda29c7-9b5c-4e7f-b8b9-65edad79e78d None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:07.203508 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda29c7-9b5c-4e7f-b8b9-65edad79e78d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:07.203783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda29c7-9b5c-4e7f-b8b9-65edad79e78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:07.223693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-0fda29c7-9b5c-4e7f-b8b9-65edad79e78d None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:07.224954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-0fda29c7-9b5c-4e7f-b8b9-65edad79e78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:07.284023 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 947/1900] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:07.293447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a615111-1ae0-4b2f-9389-95c5f9231f6c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:07.293731 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a615111-1ae0-4b2f-9389-95c5f9231f6c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:07.294007 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a615111-1ae0-4b2f-9389-95c5f9231f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:07.314880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1a615111-1ae0-4b2f-9389-95c5f9231f6c None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:07.316130 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1a615111-1ae0-4b2f-9389-95c5f9231f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:07.327232 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 954/1901] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:07.331425 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda29c7-9b5c-4e7f-b8b9-65edad79e78d None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:07.342192 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda29c7-9b5c-4e7f-b8b9-65edad79e78d None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:07.347265 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b05cf-e608-44bf-b52b-21902662a85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:07.347931 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b05cf-e608-44bf-b52b-21902662a85f admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:07.348245 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b05cf-e608-44bf-b52b-21902662a85f admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:07.348550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b05cf-e608-44bf-b52b-21902662a85f admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:07.349247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b05cf-e608-44bf-b52b-21902662a85f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:07.349558 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b05cf-e608-44bf-b52b-21902662a85f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:07.351195 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b05cf-e608-44bf-b52b-21902662a85f admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:07.362783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 948/1902] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:37:07.367277 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a615111-1ae0-4b2f-9389-95c5f9231f6c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:07.376566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a615111-1ae0-4b2f-9389-95c5f9231f6c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:07.381295 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70169b24-d830-4ee3-91fd-d09e8b2e4599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:07.381892 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70169b24-d830-4ee3-91fd-d09e8b2e4599 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:07.382176 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70169b24-d830-4ee3-91fd-d09e8b2e4599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:07.382450 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70169b24-d830-4ee3-91fd-d09e8b2e4599 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:07.383082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70169b24-d830-4ee3-91fd-d09e8b2e4599 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:07.385000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70169b24-d830-4ee3-91fd-d09e8b2e4599 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:07.542386 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 955/1903] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:07 2022] POST /identity/v3/projects => generated 363 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:07.546593 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b05cf-e608-44bf-b52b-21902662a85f admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:07.556991 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b05cf-e608-44bf-b52b-21902662a85f admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:07.624299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19388d0a-621e-4b48-aa0f-45dc8de8616a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:07.624922 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19388d0a-621e-4b48-aa0f-45dc8de8616a admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:07.625240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19388d0a-621e-4b48-aa0f-45dc8de8616a admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:07.625685 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19388d0a-621e-4b48-aa0f-45dc8de8616a admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:07.626338 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19388d0a-621e-4b48-aa0f-45dc8de8616a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:07.628287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19388d0a-621e-4b48-aa0f-45dc8de8616a admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:07.743181 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 949/1904] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:07 2022] POST /identity/v3/users => generated 282 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:07.746168 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70169b24-d830-4ee3-91fd-d09e8b2e4599 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:07.761855 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70169b24-d830-4ee3-91fd-d09e8b2e4599 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:07.772164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53df92-cb37-461c-8f10-624fc4be2a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:07.773925 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53df92-cb37-461c-8f10-624fc4be2a17 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:07.774961 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53df92-cb37-461c-8f10-624fc4be2a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:07.776141 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53df92-cb37-461c-8f10-624fc4be2a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:07.778321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53df92-cb37-461c-8f10-624fc4be2a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:07.780918 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53df92-cb37-461c-8f10-624fc4be2a17 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:07.797598 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 956/1905] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:07.801017 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19388d0a-621e-4b48-aa0f-45dc8de8616a admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:07.826147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:07.831607 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19388d0a-621e-4b48-aa0f-45dc8de8616a admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:07.839560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8140ea-d927-42fd-b05c-8edeab28e628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:07.840303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8140ea-d927-42fd-b05c-8edeab28e628 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:07.840681 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8140ea-d927-42fd-b05c-8edeab28e628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:07.841076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8140ea-d927-42fd-b05c-8edeab28e628 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:07.841830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8140ea-d927-42fd-b05c-8edeab28e628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:07.843656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8140ea-d927-42fd-b05c-8edeab28e628 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:07.856072 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 950/1906] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:07.860241 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53df92-cb37-461c-8f10-624fc4be2a17 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:07.870528 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53df92-cb37-461c-8f10-624fc4be2a17 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:07.875361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15724524-a7d2-432b-b88f-8c31af63fbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:07.875882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15724524-a7d2-432b-b88f-8c31af63fbbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:07.876163 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15724524-a7d2-432b-b88f-8c31af63fbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:07.876437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15724524-a7d2-432b-b88f-8c31af63fbbc admin admin] PATH_INFO: `/v3/projects/57242f2064944e44b17dc93a6ad87e5b/users/da0df30d5d284b7bb639656840b04cfa/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:07.911401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15724524-a7d2-432b-b88f-8c31af63fbbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=57242f2064944e44b17dc93a6ad87e5b, user_id=da0df30d5d284b7bb639656840b04cfa, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:07.914227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15724524-a7d2-432b-b88f-8c31af63fbbc admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:07.999722 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 957/1907] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:07 2022] PUT /identity/v3/projects/57242f2064944e44b17dc93a6ad87e5b/users/da0df30d5d284b7bb639656840b04cfa/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:08.016586 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2624a-abe6-4b07-a6d8-95f8ea34aab7 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:08.016896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2624a-abe6-4b07-a6d8-95f8ea34aab7 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:08.020721 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2624a-abe6-4b07-a6d8-95f8ea34aab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:08.103802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-bdb2624a-abe6-4b07-a6d8-95f8ea34aab7 None None] MFA Rules not processed for user `da0df30d5d284b7bb639656840b04cfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:08.106837 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-bdb2624a-abe6-4b07-a6d8-95f8ea34aab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:08.200078 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 951/1908] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:08.204638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15724524-a7d2-432b-b88f-8c31af63fbbc admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:08.271384 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15724524-a7d2-432b-b88f-8c31af63fbbc admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:08.276671 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62222724-5361-45d3-896d-fe33d911cfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:08.277309 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62222724-5361-45d3-896d-fe33d911cfba admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:08.277875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62222724-5361-45d3-896d-fe33d911cfba admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:08.278138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62222724-5361-45d3-896d-fe33d911cfba admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:08.278945 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62222724-5361-45d3-896d-fe33d911cfba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:08.281036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62222724-5361-45d3-896d-fe33d911cfba admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:08.399758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 958/1909] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:08 2022] POST /identity/v3/projects => generated 365 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:08.401656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2624a-abe6-4b07-a6d8-95f8ea34aab7 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:08.416117 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2624a-abe6-4b07-a6d8-95f8ea34aab7 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:08.478518 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581199b3-480a-4b0e-a386-38b3e6b77f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:08.479725 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581199b3-480a-4b0e-a386-38b3e6b77f4f admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:08.488260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581199b3-480a-4b0e-a386-38b3e6b77f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:08.488260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581199b3-480a-4b0e-a386-38b3e6b77f4f admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:08.488260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581199b3-480a-4b0e-a386-38b3e6b77f4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:08.488260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581199b3-480a-4b0e-a386-38b3e6b77f4f admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:08.576905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 952/1910] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:08 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:08.578863 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62222724-5361-45d3-896d-fe33d911cfba admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:08.590198 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62222724-5361-45d3-896d-fe33d911cfba admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:08.595097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bcd01-5389-4cfc-acef-01e278a987a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:08.595808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bcd01-5389-4cfc-acef-01e278a987a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:08.596309 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bcd01-5389-4cfc-acef-01e278a987a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:08.596740 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bcd01-5389-4cfc-acef-01e278a987a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:08.597422 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2bcd01-5389-4cfc-acef-01e278a987a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:08.599034 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2bcd01-5389-4cfc-acef-01e278a987a5 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:08.610195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 959/1911] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:08.614586 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581199b3-480a-4b0e-a386-38b3e6b77f4f admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:08.625257 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581199b3-480a-4b0e-a386-38b3e6b77f4f admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:08.633052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16edcd-3f13-4f40-a2c8-a4f081a0c20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:08.633925 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16edcd-3f13-4f40-a2c8-a4f081a0c20f admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:08.634343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16edcd-3f13-4f40-a2c8-a4f081a0c20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:08.634857 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16edcd-3f13-4f40-a2c8-a4f081a0c20f admin admin] PATH_INFO: `/v3/projects/01991c0b50bb476c8c4454418f5f635d/users/922e89fb46584ad39f2d97708c04e89d/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:08.653074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16edcd-3f13-4f40-a2c8-a4f081a0c20f admin admin] RBAC: Authorizing `identity:create_grant(project_id=01991c0b50bb476c8c4454418f5f635d, user_id=922e89fb46584ad39f2d97708c04e89d, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:08.655749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16edcd-3f13-4f40-a2c8-a4f081a0c20f admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:08.792789 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bcd01-5389-4cfc-acef-01e278a987a5 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:08.793666 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 953/1912] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:08 2022] PUT /identity/v3/projects/01991c0b50bb476c8c4454418f5f635d/users/922e89fb46584ad39f2d97708c04e89d/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:08.804234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bcd01-5389-4cfc-acef-01e278a987a5 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:08.831630 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:08.909271 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede008d7-d76b-46ad-9d98-c1365d15db1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:08.914054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede008d7-d76b-46ad-9d98-c1365d15db1c admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:08.914423 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede008d7-d76b-46ad-9d98-c1365d15db1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:08.914933 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede008d7-d76b-46ad-9d98-c1365d15db1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:08.919487 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede008d7-d76b-46ad-9d98-c1365d15db1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:08.921359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede008d7-d76b-46ad-9d98-c1365d15db1c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:08.933567 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 960/1913] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:08.937923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16edcd-3f13-4f40-a2c8-a4f081a0c20f admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:08.951685 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16edcd-3f13-4f40-a2c8-a4f081a0c20f admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:08.965917 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31b6f0-f82f-4b41-9ae8-32543815762a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:08.966604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31b6f0-f82f-4b41-9ae8-32543815762a admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:08.966862 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31b6f0-f82f-4b41-9ae8-32543815762a admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:08.967161 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31b6f0-f82f-4b41-9ae8-32543815762a admin admin] PATH_INFO: `/v3/projects/01991c0b50bb476c8c4454418f5f635d/users/922e89fb46584ad39f2d97708c04e89d/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:08.976550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31b6f0-f82f-4b41-9ae8-32543815762a admin admin] RBAC: Authorizing `identity:create_grant(project_id=01991c0b50bb476c8c4454418f5f635d, user_id=922e89fb46584ad39f2d97708c04e89d, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:08.983307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31b6f0-f82f-4b41-9ae8-32543815762a admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:08.998544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 954/1914] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:08 2022] PUT /identity/v3/projects/01991c0b50bb476c8c4454418f5f635d/users/922e89fb46584ad39f2d97708c04e89d/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:09.012831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604199d-83a2-40b4-afb3-bdc4bc18cea5 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:09.013279 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604199d-83a2-40b4-afb3-bdc4bc18cea5 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:09.013665 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604199d-83a2-40b4-afb3-bdc4bc18cea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:09.067486 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-9604199d-83a2-40b4-afb3-bdc4bc18cea5 None None] MFA Rules not processed for user `922e89fb46584ad39f2d97708c04e89d`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:09.070026 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9604199d-83a2-40b4-afb3-bdc4bc18cea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:09.191845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 961/1915] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:09.246319 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31b6f0-f82f-4b41-9ae8-32543815762a admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:09.249496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5c31b6f0-f82f-4b41-9ae8-32543815762a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:09.264792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9b841-3c99-46b1-bfed-cf7c852b01b3 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:09.265421 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9b841-3c99-46b1-bfed-cf7c852b01b3 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:09.266112 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9b841-3c99-46b1-bfed-cf7c852b01b3 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:09.269532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 962/1916] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 19 10:37:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 10:37:09.287800 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682dfff-be71-4a06-96b9-ba40477ca5fc None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:09.288567 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682dfff-be71-4a06-96b9-ba40477ca5fc None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:09.289526 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682dfff-be71-4a06-96b9-ba40477ca5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:09.326024 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4682dfff-be71-4a06-96b9-ba40477ca5fc None None] MFA Rules not processed for user `b4007794a30247c4a318af62d05e3b92`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:09.327782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4682dfff-be71-4a06-96b9-ba40477ca5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:09.364174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31b6f0-f82f-4b41-9ae8-32543815762a admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:09.368270 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5454e-27fe-4826-8470-2a864867eda3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:09.369038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5454e-27fe-4826-8470-2a864867eda3 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:09.369436 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5454e-27fe-4826-8470-2a864867eda3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:09.369789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5454e-27fe-4826-8470-2a864867eda3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:09.388915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa5454e-27fe-4826-8470-2a864867eda3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:09.391733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa5454e-27fe-4826-8470-2a864867eda3 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:09.406124 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 955/1917] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:09.500257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 963/1918] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 19 10:37:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:09.502248 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5454e-27fe-4826-8470-2a864867eda3 service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:09.504176 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4aa5454e-27fe-4826-8470-2a864867eda3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:09.526606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5454e-27fe-4826-8470-2a864867eda3 service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:09.532812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb20f1-f798-4c30-8e13-2a96cc5fdfdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:09.533620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb20f1-f798-4c30-8e13-2a96cc5fdfdf service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:09.533903 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb20f1-f798-4c30-8e13-2a96cc5fdfdf service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:09.534314 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb20f1-f798-4c30-8e13-2a96cc5fdfdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:09.542491 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682dfff-be71-4a06-96b9-ba40477ca5fc None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:09.546660 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4682dfff-be71-4a06-96b9-ba40477ca5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:09.550686 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb20f1-f798-4c30-8e13-2a96cc5fdfdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:09.552306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb20f1-f798-4c30-8e13-2a96cc5fdfdf service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:09.569985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 956/1919] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:09.646251 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682dfff-be71-4a06-96b9-ba40477ca5fc None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:09.651503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056936a-9d5a-4ed6-83b5-1bad9cfd921a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:09.652143 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056936a-9d5a-4ed6-83b5-1bad9cfd921a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:09.652428 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056936a-9d5a-4ed6-83b5-1bad9cfd921a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:09.652702 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056936a-9d5a-4ed6-83b5-1bad9cfd921a admin admin] PATH_INFO: `/v3/users/276c75cfbbeb4a17b27004fade720adb` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:09.654131 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f056936a-9d5a-4ed6-83b5-1bad9cfd921a admin admin] RBAC: Authorizing `identity:delete_user(user_id=276c75cfbbeb4a17b27004fade720adb)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:09.656087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f056936a-9d5a-4ed6-83b5-1bad9cfd921a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:09.731026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb20f1-f798-4c30-8e13-2a96cc5fdfdf service nova] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:09.746893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb20f1-f798-4c30-8e13-2a96cc5fdfdf service nova] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:09.750890 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a39a12-c638-446e-bf30-6e560b4c2e14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:09.751582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a39a12-c638-446e-bf30-6e560b4c2e14 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:09.751918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a39a12-c638-446e-bf30-6e560b4c2e14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:09.752324 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a39a12-c638-446e-bf30-6e560b4c2e14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:09.814922 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a39a12-c638-446e-bf30-6e560b4c2e14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:09.817908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a39a12-c638-446e-bf30-6e560b4c2e14 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:09.829718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 957/1920] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:37:09.913622 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f056936a-9d5a-4ed6-83b5-1bad9cfd921a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '276c75cfbbeb4a17b27004fade720adb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:09.914275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f056936a-9d5a-4ed6-83b5-1bad9cfd921a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '276c75cfbbeb4a17b27004fade720adb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:09.995602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f056936a-9d5a-4ed6-83b5-1bad9cfd921a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '276c75cfbbeb4a17b27004fade720adb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:09.997001 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f056936a-9d5a-4ed6-83b5-1bad9cfd921a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '276c75cfbbeb4a17b27004fade720adb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:10.024802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f056936a-9d5a-4ed6-83b5-1bad9cfd921a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '276c75cfbbeb4a17b27004fade720adb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:10.053129 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 964/1921] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:09 2022] DELETE /identity/v3/users/276c75cfbbeb4a17b27004fade720adb => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:10.055731 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f27ec8-3421-4d30-afe6-3a14e1b34731 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:10.056080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f27ec8-3421-4d30-afe6-3a14e1b34731 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:10.056643 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f27ec8-3421-4d30-afe6-3a14e1b34731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:10.078901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-b4f27ec8-3421-4d30-afe6-3a14e1b34731 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:10.080670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b4f27ec8-3421-4d30-afe6-3a14e1b34731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:10.166264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 958/1922] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:10.177913 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056936a-9d5a-4ed6-83b5-1bad9cfd921a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:10.179723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f056936a-9d5a-4ed6-83b5-1bad9cfd921a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:10.264561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056936a-9d5a-4ed6-83b5-1bad9cfd921a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:10.274557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdaa81-7abc-4d9a-a555-7af54fa08f81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:10.275440 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdaa81-7abc-4d9a-a555-7af54fa08f81 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:10.276404 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdaa81-7abc-4d9a-a555-7af54fa08f81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:10.276818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdaa81-7abc-4d9a-a555-7af54fa08f81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:10.289925 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecdaa81-7abc-4d9a-a555-7af54fa08f81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:10.292533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecdaa81-7abc-4d9a-a555-7af54fa08f81 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:10.312342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 965/1923] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:10.357216 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:10.864628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f27ec8-3421-4d30-afe6-3a14e1b34731 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:10.866463 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b4f27ec8-3421-4d30-afe6-3a14e1b34731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:10.885357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f27ec8-3421-4d30-afe6-3a14e1b34731 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:10.890442 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4b68a-d2ec-4b06-9934-a225f2da1745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:10.895283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4b68a-d2ec-4b06-9934-a225f2da1745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:10.895283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4b68a-d2ec-4b06-9934-a225f2da1745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:10.895283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4b68a-d2ec-4b06-9934-a225f2da1745 admin admin] PATH_INFO: `/v3/users/7f4f6efe430a45198b1adeb9f6a5a527` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:10.895283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4b68a-d2ec-4b06-9934-a225f2da1745 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f4f6efe430a45198b1adeb9f6a5a527)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:10.895283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4b68a-d2ec-4b06-9934-a225f2da1745 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:10.902683 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdaa81-7abc-4d9a-a555-7af54fa08f81 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:10.904937 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4ecdaa81-7abc-4d9a-a555-7af54fa08f81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:10.925196 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdaa81-7abc-4d9a-a555-7af54fa08f81 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:10.930218 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a73db-d03e-4374-8f24-737ececf7268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:10.930765 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a73db-d03e-4374-8f24-737ececf7268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:10.931044 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a73db-d03e-4374-8f24-737ececf7268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:10.931322 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a73db-d03e-4374-8f24-737ececf7268 admin admin] PATH_INFO: `/v3/projects/42f07e17c66a439e878a6ee0b1d93314` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:10.932439 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a73db-d03e-4374-8f24-737ececf7268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42f07e17c66a439e878a6ee0b1d93314)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:10.934377 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a73db-d03e-4374-8f24-737ececf7268 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:11.083246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f6f4b68a-d2ec-4b06-9934-a225f2da1745 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f4f6efe430a45198b1adeb9f6a5a527'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:11.083940 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f33a73db-d03e-4374-8f24-737ececf7268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42f07e17c66a439e878a6ee0b1d93314', 'request_id': 'req-f33a73db-d03e-4374-8f24-737ececf7268'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:11.090139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f33a73db-d03e-4374-8f24-737ececf7268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42f07e17c66a439e878a6ee0b1d93314', 'request_id': 'req-f33a73db-d03e-4374-8f24-737ececf7268'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:11.096481 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f33a73db-d03e-4374-8f24-737ececf7268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42f07e17c66a439e878a6ee0b1d93314', 'request_id': 'req-f33a73db-d03e-4374-8f24-737ececf7268'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:11.137331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f6f4b68a-d2ec-4b06-9934-a225f2da1745 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f4f6efe430a45198b1adeb9f6a5a527'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:11.138020 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f6f4b68a-d2ec-4b06-9934-a225f2da1745 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f4f6efe430a45198b1adeb9f6a5a527'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:11.138780 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f6f4b68a-d2ec-4b06-9934-a225f2da1745 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f4f6efe430a45198b1adeb9f6a5a527'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:11.146495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f6f4b68a-d2ec-4b06-9934-a225f2da1745 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f4f6efe430a45198b1adeb9f6a5a527'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:11.165902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 959/1924] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:10 2022] DELETE /identity/v3/users/7f4f6efe430a45198b1adeb9f6a5a527 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:11.171032 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb878cb-a6f2-487c-8f67-67300aa4fe78 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:11.171341 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb878cb-a6f2-487c-8f67-67300aa4fe78 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:11.171620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb878cb-a6f2-487c-8f67-67300aa4fe78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:11.188699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f33a73db-d03e-4374-8f24-737ececf7268 admin admin] The token cache is being invalidate because project 42f07e17c66a439e878a6ee0b1d93314 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:11.189113 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f33a73db-d03e-4374-8f24-737ececf7268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:11.194951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f33a73db-d03e-4374-8f24-737ececf7268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:11.194951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 966/1925] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:10 2022] DELETE /identity/v3/projects/42f07e17c66a439e878a6ee0b1d93314 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:11.195314 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-0bb878cb-a6f2-487c-8f67-67300aa4fe78 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:11.196171 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-0bb878cb-a6f2-487c-8f67-67300aa4fe78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:11.196486 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a73db-d03e-4374-8f24-737ececf7268 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:11.198385 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f33a73db-d03e-4374-8f24-737ececf7268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:11.262877 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 960/1926] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:11.273989 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb878cb-a6f2-487c-8f67-67300aa4fe78 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:11.275670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-0bb878cb-a6f2-487c-8f67-67300aa4fe78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:11.279367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a73db-d03e-4374-8f24-737ececf7268 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:11.285310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4144-df61-4313-878c-08772d79a8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:11.288611 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4144-df61-4313-878c-08772d79a8e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:11.288611 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4144-df61-4313-878c-08772d79a8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:11.288611 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4144-df61-4313-878c-08772d79a8e8 admin admin] PATH_INFO: `/v3/users/7b3011ebe40a42399fb6ce58fb00176c` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:11.288611 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e4144-df61-4313-878c-08772d79a8e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b3011ebe40a42399fb6ce58fb00176c)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:11.289725 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e4144-df61-4313-878c-08772d79a8e8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:11.335112 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb878cb-a6f2-487c-8f67-67300aa4fe78 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:11.339121 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938001fa-6e4c-43e9-8faf-11e68a2c5ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:11.339722 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938001fa-6e4c-43e9-8faf-11e68a2c5ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:11.339997 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938001fa-6e4c-43e9-8faf-11e68a2c5ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:11.340262 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938001fa-6e4c-43e9-8faf-11e68a2c5ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:11.349492 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938001fa-6e4c-43e9-8faf-11e68a2c5ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:11.350899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938001fa-6e4c-43e9-8faf-11e68a2c5ba2 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:11.362845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 961/1927] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:37:11.485851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3b9e4144-df61-4313-878c-08772d79a8e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b3011ebe40a42399fb6ce58fb00176c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:11.486343 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3b9e4144-df61-4313-878c-08772d79a8e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b3011ebe40a42399fb6ce58fb00176c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:11.737867 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3b9e4144-df61-4313-878c-08772d79a8e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b3011ebe40a42399fb6ce58fb00176c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:11.738649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3b9e4144-df61-4313-878c-08772d79a8e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b3011ebe40a42399fb6ce58fb00176c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:11.757672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3b9e4144-df61-4313-878c-08772d79a8e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b3011ebe40a42399fb6ce58fb00176c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:11.773422 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 967/1928] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:11 2022] DELETE /identity/v3/users/7b3011ebe40a42399fb6ce58fb00176c => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:11.889991 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:12.213835 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938001fa-6e4c-43e9-8faf-11e68a2c5ba2 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:12.213835 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-938001fa-6e4c-43e9-8faf-11e68a2c5ba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:12.233567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938001fa-6e4c-43e9-8faf-11e68a2c5ba2 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:12.310847 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df646c-9d36-40c7-b669-2ce0c3fa7609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:12.311804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df646c-9d36-40c7-b669-2ce0c3fa7609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:12.312160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df646c-9d36-40c7-b669-2ce0c3fa7609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:12.312515 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df646c-9d36-40c7-b669-2ce0c3fa7609 admin admin] PATH_INFO: `/v3/projects/5346838c646e4c64bb9b6bb48b09fd71` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:12.315904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df646c-9d36-40c7-b669-2ce0c3fa7609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5346838c646e4c64bb9b6bb48b09fd71)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:12.320251 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df646c-9d36-40c7-b669-2ce0c3fa7609 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:12.422277 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4144-df61-4313-878c-08772d79a8e8 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:12.423783 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3b9e4144-df61-4313-878c-08772d79a8e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:12.440699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4144-df61-4313-878c-08772d79a8e8 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:12.446064 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf42740-1e2b-4549-9eb4-685011978f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:12.446682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf42740-1e2b-4549-9eb4-685011978f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:12.446969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf42740-1e2b-4549-9eb4-685011978f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:12.447249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf42740-1e2b-4549-9eb4-685011978f56 admin admin] PATH_INFO: `/v3/projects/e801bccdf7114dbc8856179365584a80` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:12.448448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf42740-1e2b-4549-9eb4-685011978f56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e801bccdf7114dbc8856179365584a80)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:12.450351 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf42740-1e2b-4549-9eb4-685011978f56 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:12.522091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-63df646c-9d36-40c7-b669-2ce0c3fa7609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5346838c646e4c64bb9b6bb48b09fd71', 'request_id': 'req-63df646c-9d36-40c7-b669-2ce0c3fa7609'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:12.533046 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-63df646c-9d36-40c7-b669-2ce0c3fa7609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5346838c646e4c64bb9b6bb48b09fd71', 'request_id': 'req-63df646c-9d36-40c7-b669-2ce0c3fa7609'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:12.554277 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-63df646c-9d36-40c7-b669-2ce0c3fa7609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5346838c646e4c64bb9b6bb48b09fd71', 'request_id': 'req-63df646c-9d36-40c7-b669-2ce0c3fa7609'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:12.587111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0cf42740-1e2b-4549-9eb4-685011978f56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e801bccdf7114dbc8856179365584a80', 'request_id': 'req-0cf42740-1e2b-4549-9eb4-685011978f56'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:12.594554 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0cf42740-1e2b-4549-9eb4-685011978f56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e801bccdf7114dbc8856179365584a80', 'request_id': 'req-0cf42740-1e2b-4549-9eb4-685011978f56'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:12.601239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0cf42740-1e2b-4549-9eb4-685011978f56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e801bccdf7114dbc8856179365584a80', 'request_id': 'req-0cf42740-1e2b-4549-9eb4-685011978f56'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:12.653715 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-63df646c-9d36-40c7-b669-2ce0c3fa7609 admin admin] The token cache is being invalidate because project 5346838c646e4c64bb9b6bb48b09fd71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:12.657771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-63df646c-9d36-40c7-b669-2ce0c3fa7609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:12.657771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-63df646c-9d36-40c7-b669-2ce0c3fa7609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:12.657771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 962/1929] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:12 2022] DELETE /identity/v3/projects/5346838c646e4c64bb9b6bb48b09fd71 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:12.660027 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df646c-9d36-40c7-b669-2ce0c3fa7609 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:12.661372 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-63df646c-9d36-40c7-b669-2ce0c3fa7609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:12.729401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df646c-9d36-40c7-b669-2ce0c3fa7609 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:12.796638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0cf42740-1e2b-4549-9eb4-685011978f56 admin admin] The token cache is being invalidate because project e801bccdf7114dbc8856179365584a80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:12.797272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0cf42740-1e2b-4549-9eb4-685011978f56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:12.800105 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06930718-026a-4b98-bb50-1e873120e03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:12.800651 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0cf42740-1e2b-4549-9eb4-685011978f56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:12.801240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06930718-026a-4b98-bb50-1e873120e03e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:12.801619 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06930718-026a-4b98-bb50-1e873120e03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:12.801990 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06930718-026a-4b98-bb50-1e873120e03e admin admin] PATH_INFO: `/v3/users/bb554166f21446eb9655c277ee3edbc1` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:12.803028 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 968/1930] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:12 2022] DELETE /identity/v3/projects/e801bccdf7114dbc8856179365584a80 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:12.803826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf42740-1e2b-4549-9eb4-685011978f56 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:12.804910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06930718-026a-4b98-bb50-1e873120e03e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb554166f21446eb9655c277ee3edbc1)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:12.808320 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06930718-026a-4b98-bb50-1e873120e03e admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:12.816769 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0cf42740-1e2b-4549-9eb4-685011978f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:12.838634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf42740-1e2b-4549-9eb4-685011978f56 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:12.844737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908ed27-66e0-449e-94c3-0f2fa01cd647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:12.845564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908ed27-66e0-449e-94c3-0f2fa01cd647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:12.846013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908ed27-66e0-449e-94c3-0f2fa01cd647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:12.846553 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908ed27-66e0-449e-94c3-0f2fa01cd647 admin admin] PATH_INFO: `/v3/users/afa1974c791c48ad8cf23aeb46eb7dad` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:12.853355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908ed27-66e0-449e-94c3-0f2fa01cd647 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afa1974c791c48ad8cf23aeb46eb7dad)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:12.860303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908ed27-66e0-449e-94c3-0f2fa01cd647 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:13.024813 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-06930718-026a-4b98-bb50-1e873120e03e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb554166f21446eb9655c277ee3edbc1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:13.084778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1908ed27-66e0-449e-94c3-0f2fa01cd647 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afa1974c791c48ad8cf23aeb46eb7dad'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:13.085605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1908ed27-66e0-449e-94c3-0f2fa01cd647 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afa1974c791c48ad8cf23aeb46eb7dad'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:13.103176 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-06930718-026a-4b98-bb50-1e873120e03e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb554166f21446eb9655c277ee3edbc1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:13.104477 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-06930718-026a-4b98-bb50-1e873120e03e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb554166f21446eb9655c277ee3edbc1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:13.105171 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-06930718-026a-4b98-bb50-1e873120e03e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb554166f21446eb9655c277ee3edbc1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:13.113899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-06930718-026a-4b98-bb50-1e873120e03e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb554166f21446eb9655c277ee3edbc1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:13.131679 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 963/1931] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:12 2022] DELETE /identity/v3/users/bb554166f21446eb9655c277ee3edbc1 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:13.133327 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0f00e-3b84-46ac-9220-21ade41fcea4 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:13.133769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0f00e-3b84-46ac-9220-21ade41fcea4 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:13.134194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0f00e-3b84-46ac-9220-21ade41fcea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:13.156167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-28f0f00e-3b84-46ac-9220-21ade41fcea4 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:13.157569 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-28f0f00e-3b84-46ac-9220-21ade41fcea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:13.210702 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1908ed27-66e0-449e-94c3-0f2fa01cd647 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afa1974c791c48ad8cf23aeb46eb7dad'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:13.210702 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1908ed27-66e0-449e-94c3-0f2fa01cd647 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afa1974c791c48ad8cf23aeb46eb7dad'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:13.223952 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1908ed27-66e0-449e-94c3-0f2fa01cd647 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afa1974c791c48ad8cf23aeb46eb7dad'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:13.238765 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 969/1932] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:12 2022] DELETE /identity/v3/users/afa1974c791c48ad8cf23aeb46eb7dad => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:13.243292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40079a67-1d8c-493a-bf58-78bbf2c43bff None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:13.243841 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40079a67-1d8c-493a-bf58-78bbf2c43bff None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:13.244112 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40079a67-1d8c-493a-bf58-78bbf2c43bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:13.258663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 964/1933] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:13.267950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93b68a-1948-43f6-a8e3-613a97faba21 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:13.268230 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93b68a-1948-43f6-a8e3-613a97faba21 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:13.268785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93b68a-1948-43f6-a8e3-613a97faba21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:13.270999 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-40079a67-1d8c-493a-bf58-78bbf2c43bff None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:13.272324 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-40079a67-1d8c-493a-bf58-78bbf2c43bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:13.290811 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-af93b68a-1948-43f6-a8e3-613a97faba21 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:13.292875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-af93b68a-1948-43f6-a8e3-613a97faba21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:13.340168 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 970/1934] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:13.349880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40079a67-1d8c-493a-bf58-78bbf2c43bff None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:13.357717 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-40079a67-1d8c-493a-bf58-78bbf2c43bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:13.368931 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 965/1935] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:13.370532 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93b68a-1948-43f6-a8e3-613a97faba21 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:13.380200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93b68a-1948-43f6-a8e3-613a97faba21 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:13.386514 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6ea77-fb4d-4e5d-9f10-86d19bbfd809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:13.387195 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6ea77-fb4d-4e5d-9f10-86d19bbfd809 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:13.387660 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6ea77-fb4d-4e5d-9f10-86d19bbfd809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:13.388035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6ea77-fb4d-4e5d-9f10-86d19bbfd809 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:13.388798 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6ea77-fb4d-4e5d-9f10-86d19bbfd809 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:13.389318 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6ea77-fb4d-4e5d-9f10-86d19bbfd809 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:13.391960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6ea77-fb4d-4e5d-9f10-86d19bbfd809 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:13.411076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 966/1936] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:37:13.415588 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6ea77-fb4d-4e5d-9f10-86d19bbfd809 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:13.427869 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6ea77-fb4d-4e5d-9f10-86d19bbfd809 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:13.434404 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb7271-94dd-4c86-b854-b840fdf78099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:13.436287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb7271-94dd-4c86-b854-b840fdf78099 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:13.436860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb7271-94dd-4c86-b854-b840fdf78099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:13.437655 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb7271-94dd-4c86-b854-b840fdf78099 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:13.438407 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb7271-94dd-4c86-b854-b840fdf78099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:13.440519 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb7271-94dd-4c86-b854-b840fdf78099 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:13.446383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40079a67-1d8c-493a-bf58-78bbf2c43bff None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:13.459530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683c331-0924-4eff-8e49-66cecb3b49a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:13.459530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683c331-0924-4eff-8e49-66cecb3b49a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:13.459530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683c331-0924-4eff-8e49-66cecb3b49a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:13.459530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683c331-0924-4eff-8e49-66cecb3b49a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:13.469228 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683c331-0924-4eff-8e49-66cecb3b49a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:13.470676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683c331-0924-4eff-8e49-66cecb3b49a2 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:13.482580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 971/1937] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:37:13.547547 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 967/1938] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:13 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:37:13.551634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683c331-0924-4eff-8e49-66cecb3b49a2 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:13.561437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683c331-0924-4eff-8e49-66cecb3b49a2 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:13.625750 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba13f6b-7e3b-4e60-a506-643095535b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:13.626364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba13f6b-7e3b-4e60-a506-643095535b63 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:13.626649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba13f6b-7e3b-4e60-a506-643095535b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:13.626926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba13f6b-7e3b-4e60-a506-643095535b63 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:13.627558 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba13f6b-7e3b-4e60-a506-643095535b63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:13.629627 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba13f6b-7e3b-4e60-a506-643095535b63 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:13.667601 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb7271-94dd-4c86-b854-b840fdf78099 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:13.670041 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-57fb7271-94dd-4c86-b854-b840fdf78099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:13.685425 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb7271-94dd-4c86-b854-b840fdf78099 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:13.690996 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6078a3-078d-488d-ad0d-82e9801a0c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:13.692949 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6078a3-078d-488d-ad0d-82e9801a0c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:13.692949 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6078a3-078d-488d-ad0d-82e9801a0c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:13.692949 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6078a3-078d-488d-ad0d-82e9801a0c75 admin admin] PATH_INFO: `/v3/projects/980bf4df217d4ccfad8d808d840d414e` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:13.694138 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6078a3-078d-488d-ad0d-82e9801a0c75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=980bf4df217d4ccfad8d808d840d414e)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:13.697926 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6078a3-078d-488d-ad0d-82e9801a0c75 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:13.752991 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:13.760493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 972/1939] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:13 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:13.763786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba13f6b-7e3b-4e60-a506-643095535b63 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:13.776228 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba13f6b-7e3b-4e60-a506-643095535b63 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:13.796223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa4e16-b51d-4fb1-b293-a7edcbb5e4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:13.797004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa4e16-b51d-4fb1-b293-a7edcbb5e4ee admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:13.797456 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa4e16-b51d-4fb1-b293-a7edcbb5e4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:13.797676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa4e16-b51d-4fb1-b293-a7edcbb5e4ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:13.800253 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa4e16-b51d-4fb1-b293-a7edcbb5e4ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:13.802013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa4e16-b51d-4fb1-b293-a7edcbb5e4ee admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:13.813710 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 973/1940] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:13.817786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa4e16-b51d-4fb1-b293-a7edcbb5e4ee admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:13.832649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa4e16-b51d-4fb1-b293-a7edcbb5e4ee admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:13.836352 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea71128-4b35-46a6-84b8-f23fd2da014b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:13.837259 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea71128-4b35-46a6-84b8-f23fd2da014b admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:13.837538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea71128-4b35-46a6-84b8-f23fd2da014b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:13.837813 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea71128-4b35-46a6-84b8-f23fd2da014b admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:13.838523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea71128-4b35-46a6-84b8-f23fd2da014b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:13.840137 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea71128-4b35-46a6-84b8-f23fd2da014b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:13.853089 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 974/1941] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:13.855404 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea71128-4b35-46a6-84b8-f23fd2da014b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:13.865396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea71128-4b35-46a6-84b8-f23fd2da014b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:13.870868 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fee83-ad7a-4153-9d16-7db4bd7f5470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:13.871580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fee83-ad7a-4153-9d16-7db4bd7f5470 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:13.871952 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fee83-ad7a-4153-9d16-7db4bd7f5470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:13.872478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fee83-ad7a-4153-9d16-7db4bd7f5470 admin admin] PATH_INFO: `/v3/projects/b411a2825f6542bab753a581c78fe9ed/users/3ee15ca59e7545539898013ad4a4cffc/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:13.891226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61fee83-ad7a-4153-9d16-7db4bd7f5470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b411a2825f6542bab753a581c78fe9ed, user_id=3ee15ca59e7545539898013ad4a4cffc, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:13.893947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61fee83-ad7a-4153-9d16-7db4bd7f5470 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:13.895489 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-5c6078a3-078d-488d-ad0d-82e9801a0c75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '980bf4df217d4ccfad8d808d840d414e', 'request_id': 'req-5c6078a3-078d-488d-ad0d-82e9801a0c75'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:13.901013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-5c6078a3-078d-488d-ad0d-82e9801a0c75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '980bf4df217d4ccfad8d808d840d414e', 'request_id': 'req-5c6078a3-078d-488d-ad0d-82e9801a0c75'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:13.908981 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-5c6078a3-078d-488d-ad0d-82e9801a0c75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '980bf4df217d4ccfad8d808d840d414e', 'request_id': 'req-5c6078a3-078d-488d-ad0d-82e9801a0c75'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:14.006669 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-5c6078a3-078d-488d-ad0d-82e9801a0c75 admin admin] The token cache is being invalidate because project 980bf4df217d4ccfad8d808d840d414e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:14.006669 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-5c6078a3-078d-488d-ad0d-82e9801a0c75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:14.007057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-5c6078a3-078d-488d-ad0d-82e9801a0c75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:14.008114 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 975/1942] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:13 2022] PUT /identity/v3/projects/b411a2825f6542bab753a581c78fe9ed/users/3ee15ca59e7545539898013ad4a4cffc/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:14.008856 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fee83-ad7a-4153-9d16-7db4bd7f5470 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:14.011392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b61fee83-ad7a-4153-9d16-7db4bd7f5470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:14.018639 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 968/1943] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:13 2022] DELETE /identity/v3/projects/980bf4df217d4ccfad8d808d840d414e => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:14.024477 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1b5c0-cccd-4426-98f2-06c34add528d None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:14.024632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1b5c0-cccd-4426-98f2-06c34add528d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:14.024940 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1b5c0-cccd-4426-98f2-06c34add528d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:14.050337 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-a1a1b5c0-cccd-4426-98f2-06c34add528d None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:14.051901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-a1a1b5c0-cccd-4426-98f2-06c34add528d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:14.090273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fee83-ad7a-4153-9d16-7db4bd7f5470 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:14.097496 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201684f-e981-4b49-9edd-20932fe1aa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:14.098331 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201684f-e981-4b49-9edd-20932fe1aa4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:14.098831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201684f-e981-4b49-9edd-20932fe1aa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:14.099899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201684f-e981-4b49-9edd-20932fe1aa4e admin admin] PATH_INFO: `/v3/projects/702aec329384434b8a9494b50d113ad6` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:14.101129 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201684f-e981-4b49-9edd-20932fe1aa4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=702aec329384434b8a9494b50d113ad6)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:14.104781 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201684f-e981-4b49-9edd-20932fe1aa4e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:14.130868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 969/1944] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:14.132040 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800e985-2462-4f59-9255-ee1bfea6ed0a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:14.134223 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800e985-2462-4f59-9255-ee1bfea6ed0a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:14.134613 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800e985-2462-4f59-9255-ee1bfea6ed0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:14.190062 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-f800e985-2462-4f59-9255-ee1bfea6ed0a None None] MFA Rules not processed for user `3ee15ca59e7545539898013ad4a4cffc`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:14.191071 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-f800e985-2462-4f59-9255-ee1bfea6ed0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:14.256748 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1201684f-e981-4b49-9edd-20932fe1aa4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '702aec329384434b8a9494b50d113ad6', 'request_id': 'req-1201684f-e981-4b49-9edd-20932fe1aa4e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:14.267856 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1201684f-e981-4b49-9edd-20932fe1aa4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '702aec329384434b8a9494b50d113ad6', 'request_id': 'req-1201684f-e981-4b49-9edd-20932fe1aa4e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:14.282274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1201684f-e981-4b49-9edd-20932fe1aa4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '702aec329384434b8a9494b50d113ad6', 'request_id': 'req-1201684f-e981-4b49-9edd-20932fe1aa4e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:14.331572 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 970/1945] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:14.334728 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750c640-808e-4482-9dd7-80393a81cb39 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:14.336199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750c640-808e-4482-9dd7-80393a81cb39 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:14.336896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750c640-808e-4482-9dd7-80393a81cb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:14.385180 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-2750c640-808e-4482-9dd7-80393a81cb39 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:14.386534 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-2750c640-808e-4482-9dd7-80393a81cb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:14.388806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1201684f-e981-4b49-9edd-20932fe1aa4e admin admin] The token cache is being invalidate because project 702aec329384434b8a9494b50d113ad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:14.389133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1201684f-e981-4b49-9edd-20932fe1aa4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:14.389802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1201684f-e981-4b49-9edd-20932fe1aa4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:14.391863 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 976/1946] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:14 2022] DELETE /identity/v3/projects/702aec329384434b8a9494b50d113ad6 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:14.392640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201684f-e981-4b49-9edd-20932fe1aa4e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:14.394276 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1201684f-e981-4b49-9edd-20932fe1aa4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:14.494939 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 971/1947] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:14.495968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750c640-808e-4482-9dd7-80393a81cb39 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:14.497400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-2750c640-808e-4482-9dd7-80393a81cb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:14.503038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201684f-e981-4b49-9edd-20932fe1aa4e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:14.509517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f4408-7e82-4520-813b-9cc847506d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:14.510137 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f4408-7e82-4520-813b-9cc847506d85 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:14.510352 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750c640-808e-4482-9dd7-80393a81cb39 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:14.510471 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f4408-7e82-4520-813b-9cc847506d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:14.510684 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f4408-7e82-4520-813b-9cc847506d85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:14.511334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f4408-7e82-4520-813b-9cc847506d85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:14.513345 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f4408-7e82-4520-813b-9cc847506d85 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:14.515200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e857ca2-dbc1-4682-963e-3334d9f7fe33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:14.516087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e857ca2-dbc1-4682-963e-3334d9f7fe33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:14.516368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e857ca2-dbc1-4682-963e-3334d9f7fe33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:14.516638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e857ca2-dbc1-4682-963e-3334d9f7fe33 admin admin] PATH_INFO: `/v3/users/fcc5172457d54a3187f755416a0f7435` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:14.517796 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e857ca2-dbc1-4682-963e-3334d9f7fe33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcc5172457d54a3187f755416a0f7435)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:14.519718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e857ca2-dbc1-4682-963e-3334d9f7fe33 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:14.615025 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 977/1948] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:14 2022] POST /identity/v3/projects => generated 385 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:14.616067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f4408-7e82-4520-813b-9cc847506d85 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:14.679019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f4408-7e82-4520-813b-9cc847506d85 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:14.684014 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afd740-4c02-41f7-91d3-cc670abdbf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:14.688742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afd740-4c02-41f7-91d3-cc670abdbf40 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:14.688742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afd740-4c02-41f7-91d3-cc670abdbf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:14.688742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afd740-4c02-41f7-91d3-cc670abdbf40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:14.688742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afd740-4c02-41f7-91d3-cc670abdbf40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:14.688742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afd740-4c02-41f7-91d3-cc670abdbf40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:14.688742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afd740-4c02-41f7-91d3-cc670abdbf40 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:14.691388 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-9e857ca2-dbc1-4682-963e-3334d9f7fe33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcc5172457d54a3187f755416a0f7435'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:14.699821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 978/1949] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:37:14.700853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afd740-4c02-41f7-91d3-cc670abdbf40 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:14.710289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afd740-4c02-41f7-91d3-cc670abdbf40 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:14.715649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129b62b-c532-4577-b125-23bd5805af20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:14.717138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129b62b-c532-4577-b125-23bd5805af20 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:14.717138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129b62b-c532-4577-b125-23bd5805af20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:14.717138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129b62b-c532-4577-b125-23bd5805af20 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:14.717578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129b62b-c532-4577-b125-23bd5805af20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:14.719490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129b62b-c532-4577-b125-23bd5805af20 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:14.794121 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-9e857ca2-dbc1-4682-963e-3334d9f7fe33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcc5172457d54a3187f755416a0f7435'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:14.794792 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-9e857ca2-dbc1-4682-963e-3334d9f7fe33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcc5172457d54a3187f755416a0f7435'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:14.795458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-9e857ca2-dbc1-4682-963e-3334d9f7fe33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcc5172457d54a3187f755416a0f7435'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:14.802545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 979/1950] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:14 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:14.802545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129b62b-c532-4577-b125-23bd5805af20 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:14.803478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b129b62b-c532-4577-b125-23bd5805af20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:14.808158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-9e857ca2-dbc1-4682-963e-3334d9f7fe33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcc5172457d54a3187f755416a0f7435'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:14.815997 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129b62b-c532-4577-b125-23bd5805af20 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:14.820844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ed4d0-9a94-412b-ad08-12372b716d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:14.821844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ed4d0-9a94-412b-ad08-12372b716d24 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:14.821844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ed4d0-9a94-412b-ad08-12372b716d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:14.822052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ed4d0-9a94-412b-ad08-12372b716d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:14.823192 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ed4d0-9a94-412b-ad08-12372b716d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:14.825122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ed4d0-9a94-412b-ad08-12372b716d24 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:14.839370 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 972/1951] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:14 2022] DELETE /identity/v3/users/fcc5172457d54a3187f755416a0f7435 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:14.840002 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e857ca2-dbc1-4682-963e-3334d9f7fe33 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:14.841380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-9e857ca2-dbc1-4682-963e-3334d9f7fe33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:14.914784 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e857ca2-dbc1-4682-963e-3334d9f7fe33 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:14.920339 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a7959-0586-4f2d-9ca4-c4dde0307848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:14.924008 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a7959-0586-4f2d-9ca4-c4dde0307848 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:14.924149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 980/1952] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:14 2022] POST /identity/v3/projects => generated 393 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:14.924268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a7959-0586-4f2d-9ca4-c4dde0307848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:14.924268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a7959-0586-4f2d-9ca4-c4dde0307848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:14.924268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a7959-0586-4f2d-9ca4-c4dde0307848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:14.924268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a7959-0586-4f2d-9ca4-c4dde0307848 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:14.925516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ed4d0-9a94-412b-ad08-12372b716d24 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:14.939444 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 973/1953] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:14.940515 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a7959-0586-4f2d-9ca4-c4dde0307848 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:14.959858 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a7959-0586-4f2d-9ca4-c4dde0307848 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:14.993412 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:15.014974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ed4d0-9a94-412b-ad08-12372b716d24 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:15.021987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c20b35-7bbb-4679-aa20-483c82de5623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:15.023370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c20b35-7bbb-4679-aa20-483c82de5623 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:15.023847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c20b35-7bbb-4679-aa20-483c82de5623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:15.024278 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c20b35-7bbb-4679-aa20-483c82de5623 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:15.025203 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c20b35-7bbb-4679-aa20-483c82de5623 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:15.028312 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c20b35-7bbb-4679-aa20-483c82de5623 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:15.042002 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35d18a-3536-4945-9f9a-898bc0bc1849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:15.042748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35d18a-3536-4945-9f9a-898bc0bc1849 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:15.043164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35d18a-3536-4945-9f9a-898bc0bc1849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:15.043449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35d18a-3536-4945-9f9a-898bc0bc1849 admin admin] PATH_INFO: `/v3/projects/9f83fa9c99824fdd81bc9eb8b2d2adfb/users/d7c9bd0efc0e4748964d14ac9b6e281e/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:15.061338 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35d18a-3536-4945-9f9a-898bc0bc1849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f83fa9c99824fdd81bc9eb8b2d2adfb, user_id=d7c9bd0efc0e4748964d14ac9b6e281e, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:15.064146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35d18a-3536-4945-9f9a-898bc0bc1849 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:15.157665 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 981/1954] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:14 2022] POST /identity/v3/users => generated 297 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:15.161803 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c20b35-7bbb-4679-aa20-483c82de5623 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:15.172093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c20b35-7bbb-4679-aa20-483c82de5623 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:15.178063 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 974/1955] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:14 2022] PUT /identity/v3/projects/9f83fa9c99824fdd81bc9eb8b2d2adfb/users/d7c9bd0efc0e4748964d14ac9b6e281e/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:15.183107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35d18a-3536-4945-9f9a-898bc0bc1849 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:15.239529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6553b-5f77-45c5-b8d0-4398dd02a167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:15.240067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6553b-5f77-45c5-b8d0-4398dd02a167 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:15.240348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6553b-5f77-45c5-b8d0-4398dd02a167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:15.240672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6553b-5f77-45c5-b8d0-4398dd02a167 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:15.241794 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6553b-5f77-45c5-b8d0-4398dd02a167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:15.242891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6553b-5f77-45c5-b8d0-4398dd02a167 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:15.261332 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35d18a-3536-4945-9f9a-898bc0bc1849 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:15.263264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af695c-66b9-43aa-89d3-205bd7b537d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:15.264039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af695c-66b9-43aa-89d3-205bd7b537d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:15.264322 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af695c-66b9-43aa-89d3-205bd7b537d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:15.264602 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af695c-66b9-43aa-89d3-205bd7b537d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:15.265395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af695c-66b9-43aa-89d3-205bd7b537d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:15.267402 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af695c-66b9-43aa-89d3-205bd7b537d7 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:15.268060 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 982/1956] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:15.272087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6553b-5f77-45c5-b8d0-4398dd02a167 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:15.283255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6553b-5f77-45c5-b8d0-4398dd02a167 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:15.284797 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 975/1957] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:15.288484 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71805505-defe-4848-a6c6-68105b8b8b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:15.288861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71805505-defe-4848-a6c6-68105b8b8b8f admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:15.289145 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71805505-defe-4848-a6c6-68105b8b8b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:15.289427 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71805505-defe-4848-a6c6-68105b8b8b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:15.290069 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71805505-defe-4848-a6c6-68105b8b8b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:15.292094 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af695c-66b9-43aa-89d3-205bd7b537d7 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:15.292218 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71805505-defe-4848-a6c6-68105b8b8b8f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:15.305382 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af695c-66b9-43aa-89d3-205bd7b537d7 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:15.306538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 983/1958] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:15.307092 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71805505-defe-4848-a6c6-68105b8b8b8f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:15.311499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696616a-af87-401a-b920-6e29fcb01804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:15.312190 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696616a-af87-401a-b920-6e29fcb01804 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:15.312807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696616a-af87-401a-b920-6e29fcb01804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:15.313292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696616a-af87-401a-b920-6e29fcb01804 admin admin] PATH_INFO: `/v3/projects/9f83fa9c99824fdd81bc9eb8b2d2adfb/users/d7c9bd0efc0e4748964d14ac9b6e281e/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:15.319051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696616a-af87-401a-b920-6e29fcb01804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f83fa9c99824fdd81bc9eb8b2d2adfb, user_id=d7c9bd0efc0e4748964d14ac9b6e281e, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:15.322226 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696616a-af87-401a-b920-6e29fcb01804 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:15.336089 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71805505-defe-4848-a6c6-68105b8b8b8f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:15.355394 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 976/1959] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:15 2022] PUT /identity/v3/projects/9f83fa9c99824fdd81bc9eb8b2d2adfb/users/d7c9bd0efc0e4748964d14ac9b6e281e/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:15.357011 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696616a-af87-401a-b920-6e29fcb01804 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:15.392288 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c696616a-af87-401a-b920-6e29fcb01804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:15.422085 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3baec-b971-4823-9ebd-2d32842d463d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:15.422836 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3baec-b971-4823-9ebd-2d32842d463d admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:15.423168 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3baec-b971-4823-9ebd-2d32842d463d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:15.423450 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3baec-b971-4823-9ebd-2d32842d463d admin admin] PATH_INFO: `/v3/projects/4407ee533ae746619d73f2cc637f6c94/users/b2c536246b8746d9b43088e837b028eb/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:15.442747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3baec-b971-4823-9ebd-2d32842d463d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4407ee533ae746619d73f2cc637f6c94, user_id=b2c536246b8746d9b43088e837b028eb, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:15.445913 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3baec-b971-4823-9ebd-2d32842d463d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:15.448124 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:15.475008 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696616a-af87-401a-b920-6e29fcb01804 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:15.482141 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c65331-479f-4f25-a89a-321809a282cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:15.482848 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c65331-479f-4f25-a89a-321809a282cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:15.483190 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c65331-479f-4f25-a89a-321809a282cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:15.483554 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c65331-479f-4f25-a89a-321809a282cb admin admin] PATH_INFO: `/v3/projects/9f78d50ed7674387a07f0fbe5e432bac` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:15.484997 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c65331-479f-4f25-a89a-321809a282cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f78d50ed7674387a07f0fbe5e432bac)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:15.492862 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c65331-479f-4f25-a89a-321809a282cb admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:15.549294 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 984/1960] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:15 2022] PUT /identity/v3/projects/4407ee533ae746619d73f2cc637f6c94/users/b2c536246b8746d9b43088e837b028eb/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:15.558775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91eaa00-f9f0-4781-899b-63a1f2603e1a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:15.558775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91eaa00-f9f0-4781-899b-63a1f2603e1a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:15.558775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91eaa00-f9f0-4781-899b-63a1f2603e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:15.579245 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c91eaa00-f9f0-4781-899b-63a1f2603e1a None None] MFA Rules not processed for user `d7c9bd0efc0e4748964d14ac9b6e281e`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:15.580471 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c91eaa00-f9f0-4781-899b-63a1f2603e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:15.637651 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f7c65331-479f-4f25-a89a-321809a282cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f78d50ed7674387a07f0fbe5e432bac', 'request_id': 'req-f7c65331-479f-4f25-a89a-321809a282cb'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:15.643707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f7c65331-479f-4f25-a89a-321809a282cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f78d50ed7674387a07f0fbe5e432bac', 'request_id': 'req-f7c65331-479f-4f25-a89a-321809a282cb'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:15.651807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f7c65331-479f-4f25-a89a-321809a282cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f78d50ed7674387a07f0fbe5e432bac', 'request_id': 'req-f7c65331-479f-4f25-a89a-321809a282cb'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:15.658745 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 985/1961] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:15.660013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2d0a9-f851-4141-aed6-3bb309e3ed9a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:15.660302 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2d0a9-f851-4141-aed6-3bb309e3ed9a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:15.660615 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2d0a9-f851-4141-aed6-3bb309e3ed9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:15.688916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b0e2d0a9-f851-4141-aed6-3bb309e3ed9a None None] MFA Rules not processed for user `b2c536246b8746d9b43088e837b028eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:15.690226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b0e2d0a9-f851-4141-aed6-3bb309e3ed9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:15.745904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f7c65331-479f-4f25-a89a-321809a282cb admin admin] The token cache is being invalidate because project 9f78d50ed7674387a07f0fbe5e432bac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:15.746465 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f7c65331-479f-4f25-a89a-321809a282cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:15.747581 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f7c65331-479f-4f25-a89a-321809a282cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:15.753210 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 977/1962] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:15 2022] DELETE /identity/v3/projects/9f78d50ed7674387a07f0fbe5e432bac => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:15.753210 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c65331-479f-4f25-a89a-321809a282cb admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:15.755671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-f7c65331-479f-4f25-a89a-321809a282cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:15.756230 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 986/1963] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:15.758914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c5a6e-8acd-430b-80a0-d350a283a456 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:15.759540 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c5a6e-8acd-430b-80a0-d350a283a456 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:15.760356 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c5a6e-8acd-430b-80a0-d350a283a456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:15.782201 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-773c5a6e-8acd-430b-80a0-d350a283a456 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:15.783417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-773c5a6e-8acd-430b-80a0-d350a283a456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:15.830150 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c65331-479f-4f25-a89a-321809a282cb admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:15.842043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc0154-1d8a-4ec1-b2bc-4cff03da2029 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:15.843979 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc0154-1d8a-4ec1-b2bc-4cff03da2029 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:15.844266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc0154-1d8a-4ec1-b2bc-4cff03da2029 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:15.844549 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc0154-1d8a-4ec1-b2bc-4cff03da2029 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:15.848472 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-defc0154-1d8a-4ec1-b2bc-4cff03da2029 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:15.863757 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 987/1964] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:15.863757 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c5a6e-8acd-430b-80a0-d350a283a456 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:15.863757 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-773c5a6e-8acd-430b-80a0-d350a283a456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:15.873122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c5a6e-8acd-430b-80a0-d350a283a456 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:15.878066 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e6849-5bcd-4395-b760-8d6d5390ea56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:15.878681 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e6849-5bcd-4395-b760-8d6d5390ea56 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:15.878966 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e6849-5bcd-4395-b760-8d6d5390ea56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:15.879244 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e6849-5bcd-4395-b760-8d6d5390ea56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:15.879893 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e6849-5bcd-4395-b760-8d6d5390ea56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:15.881844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e6849-5bcd-4395-b760-8d6d5390ea56 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:15.933557 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defc0154-1d8a-4ec1-b2bc-4cff03da2029 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:15.935012 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defc0154-1d8a-4ec1-b2bc-4cff03da2029 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:15.944068 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 978/1965] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:15.945360 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3097f-13cf-4e35-99ea-1ae86bc0976b None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:15.945700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3097f-13cf-4e35-99ea-1ae86bc0976b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:15.945985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3097f-13cf-4e35-99ea-1ae86bc0976b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:15.967169 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-04c3097f-13cf-4e35-99ea-1ae86bc0976b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:15.968450 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-04c3097f-13cf-4e35-99ea-1ae86bc0976b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:15.978220 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 979/1966] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:15.983091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3097f-13cf-4e35-99ea-1ae86bc0976b None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:15.994251 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3097f-13cf-4e35-99ea-1ae86bc0976b None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:15.999659 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce3c0c-7d69-4999-9bd7-5ca6505651ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:16.000302 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce3c0c-7d69-4999-9bd7-5ca6505651ce admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:16.000584 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce3c0c-7d69-4999-9bd7-5ca6505651ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:16.000858 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce3c0c-7d69-4999-9bd7-5ca6505651ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:16.001650 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce3c0c-7d69-4999-9bd7-5ca6505651ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:16.001960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce3c0c-7d69-4999-9bd7-5ca6505651ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:16.004220 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce3c0c-7d69-4999-9bd7-5ca6505651ce admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:16.023731 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 980/1967] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:37:16.024052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 988/1968] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:15 2022] POST /identity/v3/projects => generated 393 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:16.028016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e6849-5bcd-4395-b760-8d6d5390ea56 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:16.028158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce3c0c-7d69-4999-9bd7-5ca6505651ce admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:16.054174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce3c0c-7d69-4999-9bd7-5ca6505651ce admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:16.126224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e6849-5bcd-4395-b760-8d6d5390ea56 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:16.134717 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb257ed-fb6b-44d4-b730-20b5e19905da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:16.136093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb257ed-fb6b-44d4-b730-20b5e19905da admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:16.138579 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb257ed-fb6b-44d4-b730-20b5e19905da admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:16.138995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb257ed-fb6b-44d4-b730-20b5e19905da admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:16.140187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb257ed-fb6b-44d4-b730-20b5e19905da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:16.145471 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62fbb9-d921-4ee7-8e1e-792b35035b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:16.146368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62fbb9-d921-4ee7-8e1e-792b35035b4b admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:16.147112 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62fbb9-d921-4ee7-8e1e-792b35035b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:16.147538 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62fbb9-d921-4ee7-8e1e-792b35035b4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:16.148439 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62fbb9-d921-4ee7-8e1e-792b35035b4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:16.148889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb257ed-fb6b-44d4-b730-20b5e19905da admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:16.151884 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62fbb9-d921-4ee7-8e1e-792b35035b4b admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:16.248234 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 981/1969] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:16 2022] POST /identity/v3/projects => generated 375 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:16.263215 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62fbb9-d921-4ee7-8e1e-792b35035b4b admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:16.277048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 989/1970] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:16 2022] POST /identity/v3/users => generated 303 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:16.281224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb257ed-fb6b-44d4-b730-20b5e19905da admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:16.293408 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb257ed-fb6b-44d4-b730-20b5e19905da admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:16.356411 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62fbb9-d921-4ee7-8e1e-792b35035b4b admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:16.364739 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f2d96-393d-493f-b362-0021a8603c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:16.366131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f2d96-393d-493f-b362-0021a8603c79 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:16.366450 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f2d96-393d-493f-b362-0021a8603c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:16.366782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f2d96-393d-493f-b362-0021a8603c79 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:16.367373 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f2d96-393d-493f-b362-0021a8603c79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:16.369911 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f2d96-393d-493f-b362-0021a8603c79 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:16.385713 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e117b1-50a1-49bd-804b-3172cc763a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:16.386704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e117b1-50a1-49bd-804b-3172cc763a01 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:16.387190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e117b1-50a1-49bd-804b-3172cc763a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:16.387701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e117b1-50a1-49bd-804b-3172cc763a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:16.388648 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e117b1-50a1-49bd-804b-3172cc763a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:16.390506 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e117b1-50a1-49bd-804b-3172cc763a01 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:16.404530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 990/1971] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:16.406331 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e117b1-50a1-49bd-804b-3172cc763a01 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:16.417114 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e117b1-50a1-49bd-804b-3172cc763a01 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:16.422306 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564039b0-6aee-4608-89ff-94784b2c7c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:16.422960 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564039b0-6aee-4608-89ff-94784b2c7c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:16.423334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564039b0-6aee-4608-89ff-94784b2c7c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:16.423724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564039b0-6aee-4608-89ff-94784b2c7c43 admin admin] PATH_INFO: `/v3/projects/54fdf0ac78fc46e99f7e7ca83d3e8111/users/1388345c6ce747029b6a1bfef3413c4f/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:16.442889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564039b0-6aee-4608-89ff-94784b2c7c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54fdf0ac78fc46e99f7e7ca83d3e8111, user_id=1388345c6ce747029b6a1bfef3413c4f, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:16.448323 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564039b0-6aee-4608-89ff-94784b2c7c43 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:16.469265 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 982/1972] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:16 2022] POST /identity/v3/users => generated 288 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:37:16.471495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f2d96-393d-493f-b362-0021a8603c79 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:16.482652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f2d96-393d-493f-b362-0021a8603c79 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:16.487731 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fd4f4-ea5d-4fed-aa94-67aca0ba03ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:16.488464 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fd4f4-ea5d-4fed-aa94-67aca0ba03ab admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:16.488844 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fd4f4-ea5d-4fed-aa94-67aca0ba03ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:16.489368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fd4f4-ea5d-4fed-aa94-67aca0ba03ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:16.490120 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fd4f4-ea5d-4fed-aa94-67aca0ba03ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:16.491875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fd4f4-ea5d-4fed-aa94-67aca0ba03ab admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:16.502854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 983/1973] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:16.510444 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fd4f4-ea5d-4fed-aa94-67aca0ba03ab admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:16.516134 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 991/1974] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:16 2022] PUT /identity/v3/projects/54fdf0ac78fc46e99f7e7ca83d3e8111/users/1388345c6ce747029b6a1bfef3413c4f/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:16.519863 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564039b0-6aee-4608-89ff-94784b2c7c43 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:16.523952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fd4f4-ea5d-4fed-aa94-67aca0ba03ab admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:16.603177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74b0bf-3bdf-4a77-a4d6-7c6e9478db84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:16.604423 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74b0bf-3bdf-4a77-a4d6-7c6e9478db84 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:16.604706 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74b0bf-3bdf-4a77-a4d6-7c6e9478db84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:16.604981 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74b0bf-3bdf-4a77-a4d6-7c6e9478db84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:16.605658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74b0bf-3bdf-4a77-a4d6-7c6e9478db84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:16.607435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74b0bf-3bdf-4a77-a4d6-7c6e9478db84 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:16.610131 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564039b0-6aee-4608-89ff-94784b2c7c43 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:16.617154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:16.618704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04366f-b1af-44d0-b39a-2141ea188fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:16.620167 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04366f-b1af-44d0-b39a-2141ea188fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:16.620539 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04366f-b1af-44d0-b39a-2141ea188fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:16.620877 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04366f-b1af-44d0-b39a-2141ea188fe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:16.621827 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04366f-b1af-44d0-b39a-2141ea188fe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:16.624753 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 984/1975] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:16.626263 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04366f-b1af-44d0-b39a-2141ea188fe3 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:16.627154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74b0bf-3bdf-4a77-a4d6-7c6e9478db84 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:16.637591 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74b0bf-3bdf-4a77-a4d6-7c6e9478db84 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:16.642277 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 992/1976] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:16.643273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04366f-b1af-44d0-b39a-2141ea188fe3 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:16.643748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a4e34-e69e-456c-84ce-d7115988c719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:16.644446 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a4e34-e69e-456c-84ce-d7115988c719 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:16.644727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a4e34-e69e-456c-84ce-d7115988c719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:16.645009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a4e34-e69e-456c-84ce-d7115988c719 admin admin] PATH_INFO: `/v3/projects/c449e87bba034a3abedcf9290e4652c8/users/97a12c94c8884744a6c1f7f349e60ea1/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:16.655098 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04366f-b1af-44d0-b39a-2141ea188fe3 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:16.660866 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b6fda-da2b-4295-902f-b7b886b21b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:16.662373 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b6fda-da2b-4295-902f-b7b886b21b2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:16.662799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b6fda-da2b-4295-902f-b7b886b21b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:16.663185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b6fda-da2b-4295-902f-b7b886b21b2f admin admin] PATH_INFO: `/v3/projects/54fdf0ac78fc46e99f7e7ca83d3e8111/users/1388345c6ce747029b6a1bfef3413c4f/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:16.664911 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a4e34-e69e-456c-84ce-d7115988c719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c449e87bba034a3abedcf9290e4652c8, user_id=97a12c94c8884744a6c1f7f349e60ea1, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:16.665570 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b6fda-da2b-4295-902f-b7b886b21b2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=54fdf0ac78fc46e99f7e7ca83d3e8111, user_id=1388345c6ce747029b6a1bfef3413c4f, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:16.668273 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a4e34-e69e-456c-84ce-d7115988c719 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:16.668988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b6fda-da2b-4295-902f-b7b886b21b2f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:16.692113 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 993/1977] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:16 2022] PUT /identity/v3/projects/54fdf0ac78fc46e99f7e7ca83d3e8111/users/1388345c6ce747029b6a1bfef3413c4f/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:16.693712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee8cb9-e3e5-4cdf-870a-e016591b3252 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:16.694000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee8cb9-e3e5-4cdf-870a-e016591b3252 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:16.694277 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee8cb9-e3e5-4cdf-870a-e016591b3252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:16.727604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c8ee8cb9-e3e5-4cdf-870a-e016591b3252 None None] MFA Rules not processed for user `1388345c6ce747029b6a1bfef3413c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:16.730101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c8ee8cb9-e3e5-4cdf-870a-e016591b3252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:16.774654 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 985/1978] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:16 2022] PUT /identity/v3/projects/c449e87bba034a3abedcf9290e4652c8/users/97a12c94c8884744a6c1f7f349e60ea1/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:16.786115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912440e9-e35e-4d88-859e-169d4bd75975 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:16.786487 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912440e9-e35e-4d88-859e-169d4bd75975 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:16.786827 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912440e9-e35e-4d88-859e-169d4bd75975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:16.808787 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 994/1979] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:16 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:16.823927 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-912440e9-e35e-4d88-859e-169d4bd75975 None None] MFA Rules not processed for user `97a12c94c8884744a6c1f7f349e60ea1`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:16.825882 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-912440e9-e35e-4d88-859e-169d4bd75975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:16.828394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee8cb9-e3e5-4cdf-870a-e016591b3252 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:16.915402 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee8cb9-e3e5-4cdf-870a-e016591b3252 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:16.930484 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 986/1980] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:16.943595 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912440e9-e35e-4d88-859e-169d4bd75975 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:16.945411 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-912440e9-e35e-4d88-859e-169d4bd75975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:17.026184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81072c72-911e-4076-9f48-5f3ad552c07b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:17.026729 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81072c72-911e-4076-9f48-5f3ad552c07b service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:17.027012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81072c72-911e-4076-9f48-5f3ad552c07b service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:17.027292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81072c72-911e-4076-9f48-5f3ad552c07b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:17.102887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912440e9-e35e-4d88-859e-169d4bd75975 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:17.102887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087186d-1e86-4ffa-a0ea-3a2996520529 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a40b1ed93a6b4e5fb0e8f907627379a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:17.102887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087186d-1e86-4ffa-a0ea-3a2996520529 service glance] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:17.102887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087186d-1e86-4ffa-a0ea-3a2996520529 service glance] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:17.103616 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087186d-1e86-4ffa-a0ea-3a2996520529 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:17.109173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087186d-1e86-4ffa-a0ea-3a2996520529 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:17.110775 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087186d-1e86-4ffa-a0ea-3a2996520529 service glance] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:17.119430 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81072c72-911e-4076-9f48-5f3ad552c07b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:17.120979 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81072c72-911e-4076-9f48-5f3ad552c07b service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:17.122225 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 987/1981] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:37:17.144741 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 995/1982] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:16 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:17.174538 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:DELETE=4,keystone:SELECT=96 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:17.199264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087186d-1e86-4ffa-a0ea-3a2996520529 service glance] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:17.223012 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087186d-1e86-4ffa-a0ea-3a2996520529 service glance] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:17.230535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710ce3d-b435-4462-a151-911da8ea585a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:17.231075 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710ce3d-b435-4462-a151-911da8ea585a service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:17.231367 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710ce3d-b435-4462-a151-911da8ea585a service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:17.231902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710ce3d-b435-4462-a151-911da8ea585a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:17.234396 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-1710ce3d-b435-4462-a151-911da8ea585a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:17.334490 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710ce3d-b435-4462-a151-911da8ea585a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:17.336096 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710ce3d-b435-4462-a151-911da8ea585a service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:17.348708 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 988/1983] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:18.320929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:18.503811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:20.487879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:20.999840 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:22.303347 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:22.716295 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:22.831414 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:23.424615 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81072c72-911e-4076-9f48-5f3ad552c07b service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:23.489460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81072c72-911e-4076-9f48-5f3ad552c07b service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:23.495081 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035e76c-937d-4dc5-8b4e-39f64b90ce7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:23.495579 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035e76c-937d-4dc5-8b4e-39f64b90ce7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:23.495869 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035e76c-937d-4dc5-8b4e-39f64b90ce7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:23.496141 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035e76c-937d-4dc5-8b4e-39f64b90ce7e admin admin] PATH_INFO: `/v3/users/b2c536246b8746d9b43088e837b028eb` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:23.497326 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3035e76c-937d-4dc5-8b4e-39f64b90ce7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2c536246b8746d9b43088e837b028eb)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:23.499410 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3035e76c-937d-4dc5-8b4e-39f64b90ce7e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:23.594728 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710ce3d-b435-4462-a151-911da8ea585a service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:23.595284 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-1710ce3d-b435-4462-a151-911da8ea585a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:23.608008 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710ce3d-b435-4462-a151-911da8ea585a service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:23.611946 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533569b7-8ecf-4ab1-bf67-fdabb7f18d94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:23.612633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533569b7-8ecf-4ab1-bf67-fdabb7f18d94 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:23.612908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533569b7-8ecf-4ab1-bf67-fdabb7f18d94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:23.613219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533569b7-8ecf-4ab1-bf67-fdabb7f18d94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:23.614974 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-533569b7-8ecf-4ab1-bf67-fdabb7f18d94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:23.679382 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533569b7-8ecf-4ab1-bf67-fdabb7f18d94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:23.680934 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533569b7-8ecf-4ab1-bf67-fdabb7f18d94 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:23.690821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 989/1984] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:23 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:37:23.740450 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3035e76c-937d-4dc5-8b4e-39f64b90ce7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2c536246b8746d9b43088e837b028eb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:23.741222 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3035e76c-937d-4dc5-8b4e-39f64b90ce7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2c536246b8746d9b43088e837b028eb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:23.867967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3035e76c-937d-4dc5-8b4e-39f64b90ce7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2c536246b8746d9b43088e837b028eb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:23.868640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3035e76c-937d-4dc5-8b4e-39f64b90ce7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2c536246b8746d9b43088e837b028eb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:23.881516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3035e76c-937d-4dc5-8b4e-39f64b90ce7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2c536246b8746d9b43088e837b028eb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:23.894727 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 996/1985] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:23 2022] DELETE /identity/v3/users/b2c536246b8746d9b43088e837b028eb => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:23.899095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470d089-708c-4bf0-a4af-c8839760f709 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:23.899615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470d089-708c-4bf0-a4af-c8839760f709 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:23.899753 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470d089-708c-4bf0-a4af-c8839760f709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:23.922707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-c470d089-708c-4bf0-a4af-c8839760f709 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:23.923955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c470d089-708c-4bf0-a4af-c8839760f709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:23.984157 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 990/1986] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:23.993942 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035e76c-937d-4dc5-8b4e-39f64b90ce7e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:23.995571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3035e76c-937d-4dc5-8b4e-39f64b90ce7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:24.053193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035e76c-937d-4dc5-8b4e-39f64b90ce7e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:24.056814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e124e-4e3a-432a-8bd9-17bdb8b31d7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:24.057458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e124e-4e3a-432a-8bd9-17bdb8b31d7b service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:24.057734 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e124e-4e3a-432a-8bd9-17bdb8b31d7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:24.058023 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e124e-4e3a-432a-8bd9-17bdb8b31d7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:24.066946 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e124e-4e3a-432a-8bd9-17bdb8b31d7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:24.068384 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e124e-4e3a-432a-8bd9-17bdb8b31d7b service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:24.078722 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 997/1987] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:24.564353 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470d089-708c-4bf0-a4af-c8839760f709 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:24.566032 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c470d089-708c-4bf0-a4af-c8839760f709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:24.582576 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470d089-708c-4bf0-a4af-c8839760f709 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:24.588316 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d382d70-e19e-4a4f-9938-919180cedcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:24.588964 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d382d70-e19e-4a4f-9938-919180cedcd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:24.589291 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d382d70-e19e-4a4f-9938-919180cedcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:24.589567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d382d70-e19e-4a4f-9938-919180cedcd8 admin admin] PATH_INFO: `/v3/projects/4407ee533ae746619d73f2cc637f6c94` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:24.590815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d382d70-e19e-4a4f-9938-919180cedcd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4407ee533ae746619d73f2cc637f6c94)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:24.592744 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d382d70-e19e-4a4f-9938-919180cedcd8 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:24.659696 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e124e-4e3a-432a-8bd9-17bdb8b31d7b service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:24.661401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5f4e124e-4e3a-432a-8bd9-17bdb8b31d7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:24.674119 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e124e-4e3a-432a-8bd9-17bdb8b31d7b service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:24.679264 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f13dab-a62d-4ac2-93e5-3560b4c36718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:24.679906 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f13dab-a62d-4ac2-93e5-3560b4c36718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:24.686107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f13dab-a62d-4ac2-93e5-3560b4c36718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:24.686107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f13dab-a62d-4ac2-93e5-3560b4c36718 admin admin] PATH_INFO: `/v3/users/da0df30d5d284b7bb639656840b04cfa` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:24.686107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f13dab-a62d-4ac2-93e5-3560b4c36718 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da0df30d5d284b7bb639656840b04cfa)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:24.686107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f13dab-a62d-4ac2-93e5-3560b4c36718 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:24.754182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-4d382d70-e19e-4a4f-9938-919180cedcd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4407ee533ae746619d73f2cc637f6c94', 'request_id': 'req-4d382d70-e19e-4a4f-9938-919180cedcd8'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:24.761430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-4d382d70-e19e-4a4f-9938-919180cedcd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4407ee533ae746619d73f2cc637f6c94', 'request_id': 'req-4d382d70-e19e-4a4f-9938-919180cedcd8'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:24.769506 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-4d382d70-e19e-4a4f-9938-919180cedcd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4407ee533ae746619d73f2cc637f6c94', 'request_id': 'req-4d382d70-e19e-4a4f-9938-919180cedcd8'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:24.830987 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:24.838322 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-4d382d70-e19e-4a4f-9938-919180cedcd8 admin admin] The token cache is being invalidate because project 4407ee533ae746619d73f2cc637f6c94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:24.838624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-4d382d70-e19e-4a4f-9938-919180cedcd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:24.839277 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-4d382d70-e19e-4a4f-9938-919180cedcd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:24.841174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 991/1988] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:24 2022] DELETE /identity/v3/projects/4407ee533ae746619d73f2cc637f6c94 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:24.844633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d382d70-e19e-4a4f-9938-919180cedcd8 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:24.846149 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4d382d70-e19e-4a4f-9938-919180cedcd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:24.847275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-19f13dab-a62d-4ac2-93e5-3560b4c36718 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da0df30d5d284b7bb639656840b04cfa'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:24.847987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-19f13dab-a62d-4ac2-93e5-3560b4c36718 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da0df30d5d284b7bb639656840b04cfa'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:24.918356 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d382d70-e19e-4a4f-9938-919180cedcd8 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:24.923706 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30215e06-86a8-4497-9c32-752908ba75ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:24.924534 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30215e06-86a8-4497-9c32-752908ba75ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:24.924769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30215e06-86a8-4497-9c32-752908ba75ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:24.925173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30215e06-86a8-4497-9c32-752908ba75ee admin admin] PATH_INFO: `/v3/users/1388345c6ce747029b6a1bfef3413c4f` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:24.926448 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30215e06-86a8-4497-9c32-752908ba75ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=1388345c6ce747029b6a1bfef3413c4f)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:24.935409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30215e06-86a8-4497-9c32-752908ba75ee admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:24.937817 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-19f13dab-a62d-4ac2-93e5-3560b4c36718 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da0df30d5d284b7bb639656840b04cfa'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:24.939961 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-19f13dab-a62d-4ac2-93e5-3560b4c36718 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da0df30d5d284b7bb639656840b04cfa'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:24.943990 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:24.955599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-19f13dab-a62d-4ac2-93e5-3560b4c36718 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da0df30d5d284b7bb639656840b04cfa'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:24.972341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 998/1989] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:24 2022] DELETE /identity/v3/users/da0df30d5d284b7bb639656840b04cfa => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:24.976097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf3c6d-4337-4fdc-9132-9548964fd182 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:24.976097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf3c6d-4337-4fdc-9132-9548964fd182 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:24.976097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf3c6d-4337-4fdc-9132-9548964fd182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:24.995176 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-0aaf3c6d-4337-4fdc-9132-9548964fd182 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:24.996403 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0aaf3c6d-4337-4fdc-9132-9548964fd182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:25.063479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 999/1990] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:25.074008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf3c6d-4337-4fdc-9132-9548964fd182 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:25.075548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0aaf3c6d-4337-4fdc-9132-9548964fd182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:25.122175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-30215e06-86a8-4497-9c32-752908ba75ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1388345c6ce747029b6a1bfef3413c4f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:25.135486 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf3c6d-4337-4fdc-9132-9548964fd182 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:25.139128 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5e010-bf0f-47f8-8c46-1d6fa1b3ec01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:25.139742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5e010-bf0f-47f8-8c46-1d6fa1b3ec01 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:25.140079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5e010-bf0f-47f8-8c46-1d6fa1b3ec01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:25.140297 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5e010-bf0f-47f8-8c46-1d6fa1b3ec01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:25.150886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5e010-bf0f-47f8-8c46-1d6fa1b3ec01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:25.152312 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5e010-bf0f-47f8-8c46-1d6fa1b3ec01 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:25.161944 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1000/1991] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:37:25.190141 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-30215e06-86a8-4497-9c32-752908ba75ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1388345c6ce747029b6a1bfef3413c4f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:25.190824 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-30215e06-86a8-4497-9c32-752908ba75ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1388345c6ce747029b6a1bfef3413c4f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:25.191443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-30215e06-86a8-4497-9c32-752908ba75ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1388345c6ce747029b6a1bfef3413c4f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:25.199444 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-30215e06-86a8-4497-9c32-752908ba75ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1388345c6ce747029b6a1bfef3413c4f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:25.220127 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 992/1992] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:24 2022] DELETE /identity/v3/users/1388345c6ce747029b6a1bfef3413c4f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:25.652903 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5e010-bf0f-47f8-8c46-1d6fa1b3ec01 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:25.654666 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b5c5e010-bf0f-47f8-8c46-1d6fa1b3ec01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:25.675641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5e010-bf0f-47f8-8c46-1d6fa1b3ec01 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:25.686899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30215e06-86a8-4497-9c32-752908ba75ee admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:25.689044 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-30215e06-86a8-4497-9c32-752908ba75ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:25.766657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:25.770608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e099e6c-c682-46b4-88ae-591b5d5da755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:25.771283 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e099e6c-c682-46b4-88ae-591b5d5da755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:25.775353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e099e6c-c682-46b4-88ae-591b5d5da755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:25.775353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e099e6c-c682-46b4-88ae-591b5d5da755 admin admin] PATH_INFO: `/v3/projects/57242f2064944e44b17dc93a6ad87e5b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:25.775353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e099e6c-c682-46b4-88ae-591b5d5da755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57242f2064944e44b17dc93a6ad87e5b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:25.775353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e099e6c-c682-46b4-88ae-591b5d5da755 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:25.780364 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30215e06-86a8-4497-9c32-752908ba75ee admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:25.785555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4973dd9-7c7c-43da-b260-f500498161a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:25.786310 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4973dd9-7c7c-43da-b260-f500498161a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:25.786694 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4973dd9-7c7c-43da-b260-f500498161a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:25.787087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4973dd9-7c7c-43da-b260-f500498161a7 admin admin] PATH_INFO: `/v3/projects/54fdf0ac78fc46e99f7e7ca83d3e8111` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:25.788511 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4973dd9-7c7c-43da-b260-f500498161a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54fdf0ac78fc46e99f7e7ca83d3e8111)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:25.790874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4973dd9-7c7c-43da-b260-f500498161a7 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:25.911834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:25.929624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-c4973dd9-7c7c-43da-b260-f500498161a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54fdf0ac78fc46e99f7e7ca83d3e8111', 'request_id': 'req-c4973dd9-7c7c-43da-b260-f500498161a7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:25.931497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7e099e6c-c682-46b4-88ae-591b5d5da755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57242f2064944e44b17dc93a6ad87e5b', 'request_id': 'req-7e099e6c-c682-46b4-88ae-591b5d5da755'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:25.935892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-c4973dd9-7c7c-43da-b260-f500498161a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54fdf0ac78fc46e99f7e7ca83d3e8111', 'request_id': 'req-c4973dd9-7c7c-43da-b260-f500498161a7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:25.938085 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7e099e6c-c682-46b4-88ae-591b5d5da755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57242f2064944e44b17dc93a6ad87e5b', 'request_id': 'req-7e099e6c-c682-46b4-88ae-591b5d5da755'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:25.942524 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-c4973dd9-7c7c-43da-b260-f500498161a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54fdf0ac78fc46e99f7e7ca83d3e8111', 'request_id': 'req-c4973dd9-7c7c-43da-b260-f500498161a7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:25.946254 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7e099e6c-c682-46b4-88ae-591b5d5da755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57242f2064944e44b17dc93a6ad87e5b', 'request_id': 'req-7e099e6c-c682-46b4-88ae-591b5d5da755'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:26.099855 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7e099e6c-c682-46b4-88ae-591b5d5da755 admin admin] The token cache is being invalidate because project 57242f2064944e44b17dc93a6ad87e5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:26.100151 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7e099e6c-c682-46b4-88ae-591b5d5da755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:26.100774 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7e099e6c-c682-46b4-88ae-591b5d5da755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:26.102279 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1001/1993] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:25 2022] DELETE /identity/v3/projects/57242f2064944e44b17dc93a6ad87e5b => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:26.106784 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e099e6c-c682-46b4-88ae-591b5d5da755 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:26.108081 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7e099e6c-c682-46b4-88ae-591b5d5da755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:26.170633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e099e6c-c682-46b4-88ae-591b5d5da755 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:26.175295 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e07cdd-b410-46ef-85dd-dc238f11f6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:26.175954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e07cdd-b410-46ef-85dd-dc238f11f6b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:26.176236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e07cdd-b410-46ef-85dd-dc238f11f6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:26.176509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e07cdd-b410-46ef-85dd-dc238f11f6b2 admin admin] PATH_INFO: `/v3/users/922e89fb46584ad39f2d97708c04e89d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:26.177774 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e07cdd-b410-46ef-85dd-dc238f11f6b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=922e89fb46584ad39f2d97708c04e89d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:26.179633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e07cdd-b410-46ef-85dd-dc238f11f6b2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:26.204125 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-c4973dd9-7c7c-43da-b260-f500498161a7 admin admin] The token cache is being invalidate because project 54fdf0ac78fc46e99f7e7ca83d3e8111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:26.204642 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-c4973dd9-7c7c-43da-b260-f500498161a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:26.205457 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-c4973dd9-7c7c-43da-b260-f500498161a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:26.207414 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 993/1994] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:25 2022] DELETE /identity/v3/projects/54fdf0ac78fc46e99f7e7ca83d3e8111 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:26.211875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94734080-202c-44bf-83ac-40e6ae019e6d None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:26.212158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94734080-202c-44bf-83ac-40e6ae019e6d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:26.212451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94734080-202c-44bf-83ac-40e6ae019e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:26.233237 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-94734080-202c-44bf-83ac-40e6ae019e6d None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:26.234349 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-94734080-202c-44bf-83ac-40e6ae019e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:26.242883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 994/1995] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:26.252730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32b322-209f-40f9-9ece-a4d8d4834455 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:26.253025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32b322-209f-40f9-9ece-a4d8d4834455 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:26.260336 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32b322-209f-40f9-9ece-a4d8d4834455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:26.273703 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-7a32b322-209f-40f9-9ece-a4d8d4834455 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:26.274942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-7a32b322-209f-40f9-9ece-a4d8d4834455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:26.282906 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 995/1996] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:26.287389 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32b322-209f-40f9-9ece-a4d8d4834455 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:26.296314 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32b322-209f-40f9-9ece-a4d8d4834455 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:26.300962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b387051-5807-442b-9901-ec400d097949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:26.301579 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b387051-5807-442b-9901-ec400d097949 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:26.301859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b387051-5807-442b-9901-ec400d097949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:26.302148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b387051-5807-442b-9901-ec400d097949 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:26.302776 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b387051-5807-442b-9901-ec400d097949 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:26.303075 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b387051-5807-442b-9901-ec400d097949 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:26.304651 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b387051-5807-442b-9901-ec400d097949 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:26.316985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 996/1997] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:37:26.320978 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b387051-5807-442b-9901-ec400d097949 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:26.330593 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b387051-5807-442b-9901-ec400d097949 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:26.335611 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a49f5-71d9-43a7-a2c7-95748ce0e518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:26.336183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a49f5-71d9-43a7-a2c7-95748ce0e518 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:26.336547 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a49f5-71d9-43a7-a2c7-95748ce0e518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:26.336840 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a49f5-71d9-43a7-a2c7-95748ce0e518 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:26.337636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a49f5-71d9-43a7-a2c7-95748ce0e518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:26.339515 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a49f5-71d9-43a7-a2c7-95748ce0e518 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:26.352499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-69e07cdd-b410-46ef-85dd-dc238f11f6b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '922e89fb46584ad39f2d97708c04e89d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:26.353900 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-69e07cdd-b410-46ef-85dd-dc238f11f6b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '922e89fb46584ad39f2d97708c04e89d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:26.433940 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-69e07cdd-b410-46ef-85dd-dc238f11f6b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '922e89fb46584ad39f2d97708c04e89d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:26.434525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-69e07cdd-b410-46ef-85dd-dc238f11f6b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '922e89fb46584ad39f2d97708c04e89d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:26.444319 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 997/1998] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:26 2022] POST /identity/v3/projects => generated 401 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:37:26.445396 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a49f5-71d9-43a7-a2c7-95748ce0e518 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:26.447006 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-976a49f5-71d9-43a7-a2c7-95748ce0e518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:26.447340 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-69e07cdd-b410-46ef-85dd-dc238f11f6b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '922e89fb46584ad39f2d97708c04e89d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:26.459554 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1002/1999] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:26 2022] DELETE /identity/v3/users/922e89fb46584ad39f2d97708c04e89d => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:26.510929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a49f5-71d9-43a7-a2c7-95748ce0e518 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:26.571781 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d9c76-5d72-4a0b-803c-bcc414c2255d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:26.572418 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d9c76-5d72-4a0b-803c-bcc414c2255d admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:26.572702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d9c76-5d72-4a0b-803c-bcc414c2255d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:26.572988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d9c76-5d72-4a0b-803c-bcc414c2255d admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:26.573683 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d9c76-5d72-4a0b-803c-bcc414c2255d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:26.575631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d9c76-5d72-4a0b-803c-bcc414c2255d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:26.738952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 998/2000] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:26 2022] POST /identity/v3/users => generated 301 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:37:26.740402 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e07cdd-b410-46ef-85dd-dc238f11f6b2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:26.750705 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e07cdd-b410-46ef-85dd-dc238f11f6b2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:26.758086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c59c1-4838-4893-904d-0a967c4d62bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:26.759391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c59c1-4838-4893-904d-0a967c4d62bb admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:26.759684 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c59c1-4838-4893-904d-0a967c4d62bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:26.759963 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c59c1-4838-4893-904d-0a967c4d62bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:26.760666 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c59c1-4838-4893-904d-0a967c4d62bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:26.762353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c59c1-4838-4893-904d-0a967c4d62bb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:26.778776 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1003/2001] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:26.780185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d9c76-5d72-4a0b-803c-bcc414c2255d admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:26.791175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d9c76-5d72-4a0b-803c-bcc414c2255d admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:26.796267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce4eb3-b872-495d-9897-5f218b45c115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:26.796940 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce4eb3-b872-495d-9897-5f218b45c115 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:26.797292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce4eb3-b872-495d-9897-5f218b45c115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:26.797702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce4eb3-b872-495d-9897-5f218b45c115 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:26.798353 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce4eb3-b872-495d-9897-5f218b45c115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:26.799975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce4eb3-b872-495d-9897-5f218b45c115 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:26.813183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 999/2002] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:26.815537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c59c1-4838-4893-904d-0a967c4d62bb admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:26.941465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c59c1-4838-4893-904d-0a967c4d62bb admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:26.941465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5afd4-b78e-4b5e-9b64-00cbda8491ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:26.941465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5afd4-b78e-4b5e-9b64-00cbda8491ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:26.941465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5afd4-b78e-4b5e-9b64-00cbda8491ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:26.941857 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5afd4-b78e-4b5e-9b64-00cbda8491ae admin admin] PATH_INFO: `/v3/projects/fb1fe081e60a494082806e3d8aa6fed7/users/c9fa148051334dc9a8c781f661225ba4/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:26.941857 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df5afd4-b78e-4b5e-9b64-00cbda8491ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb1fe081e60a494082806e3d8aa6fed7, user_id=c9fa148051334dc9a8c781f661225ba4, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:26.941857 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df5afd4-b78e-4b5e-9b64-00cbda8491ae admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:26.997267 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1004/2003] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:26 2022] PUT /identity/v3/projects/fb1fe081e60a494082806e3d8aa6fed7/users/c9fa148051334dc9a8c781f661225ba4/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:26.998512 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce4eb3-b872-495d-9897-5f218b45c115 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:27.000491 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c1ce4eb3-b872-495d-9897-5f218b45c115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:27.015687 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6e6e1-987a-437f-9a18-b2c15ca65b0f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:27.015687 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6e6e1-987a-437f-9a18-b2c15ca65b0f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:27.015687 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6e6e1-987a-437f-9a18-b2c15ca65b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:27.021638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce4eb3-b872-495d-9897-5f218b45c115 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:27.065534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4fd6e6e1-987a-437f-9a18-b2c15ca65b0f None None] MFA Rules not processed for user `c9fa148051334dc9a8c781f661225ba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:27.065734 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:27.065845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4fd6e6e1-987a-437f-9a18-b2c15ca65b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:27.112013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4523d9-e3a4-4afd-ab30-48f30668a667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:27.112656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4523d9-e3a4-4afd-ab30-48f30668a667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:27.113000 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4523d9-e3a4-4afd-ab30-48f30668a667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:27.119048 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4523d9-e3a4-4afd-ab30-48f30668a667 admin admin] PATH_INFO: `/v3/projects/01991c0b50bb476c8c4454418f5f635d` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:27.119048 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4523d9-e3a4-4afd-ab30-48f30668a667 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01991c0b50bb476c8c4454418f5f635d)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:27.119048 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4523d9-e3a4-4afd-ab30-48f30668a667 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:27.135096 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1005/2004] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:27 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:27.139821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6e6e1-987a-437f-9a18-b2c15ca65b0f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:27.151308 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6e6e1-987a-437f-9a18-b2c15ca65b0f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:27.155920 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc4513-f2d2-410c-a377-39596e766d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:27.156521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc4513-f2d2-410c-a377-39596e766d44 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:27.156799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc4513-f2d2-410c-a377-39596e766d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:27.157094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc4513-f2d2-410c-a377-39596e766d44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:27.157727 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc4513-f2d2-410c-a377-39596e766d44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:27.159709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc4513-f2d2-410c-a377-39596e766d44 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:27.233125 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-4f4523d9-e3a4-4afd-ab30-48f30668a667 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01991c0b50bb476c8c4454418f5f635d', 'request_id': 'req-4f4523d9-e3a4-4afd-ab30-48f30668a667'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:27.238926 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-4f4523d9-e3a4-4afd-ab30-48f30668a667 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01991c0b50bb476c8c4454418f5f635d', 'request_id': 'req-4f4523d9-e3a4-4afd-ab30-48f30668a667'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:27.247817 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-4f4523d9-e3a4-4afd-ab30-48f30668a667 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01991c0b50bb476c8c4454418f5f635d', 'request_id': 'req-4f4523d9-e3a4-4afd-ab30-48f30668a667'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:27.252851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1006/2005] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:27 2022] POST /identity/v3/projects => generated 399 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:37:27.257107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc4513-f2d2-410c-a377-39596e766d44 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:27.320087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc4513-f2d2-410c-a377-39596e766d44 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:27.325680 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6ab79-6ec0-455f-9009-00a4ebc9af68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:27.328348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6ab79-6ec0-455f-9009-00a4ebc9af68 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:27.328348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6ab79-6ec0-455f-9009-00a4ebc9af68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:27.328348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6ab79-6ec0-455f-9009-00a4ebc9af68 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:27.328348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d6ab79-6ec0-455f-9009-00a4ebc9af68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:27.329486 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d6ab79-6ec0-455f-9009-00a4ebc9af68 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:27.353296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-4f4523d9-e3a4-4afd-ab30-48f30668a667 admin admin] The token cache is being invalidate because project 01991c0b50bb476c8c4454418f5f635d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:27.353619 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-4f4523d9-e3a4-4afd-ab30-48f30668a667 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:27.354260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-4f4523d9-e3a4-4afd-ab30-48f30668a667 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:27.355842 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1000/2006] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:26 2022] DELETE /identity/v3/projects/01991c0b50bb476c8c4454418f5f635d => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:27.360442 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951474b2-31ce-44dd-8f2e-b5ba1f44fc4a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:27.360727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951474b2-31ce-44dd-8f2e-b5ba1f44fc4a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:27.362734 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951474b2-31ce-44dd-8f2e-b5ba1f44fc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:27.381176 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-951474b2-31ce-44dd-8f2e-b5ba1f44fc4a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:27.382367 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-951474b2-31ce-44dd-8f2e-b5ba1f44fc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:27.391198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1001/2007] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:27.398582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e690d2-8f32-412e-8749-db0699fe33eb None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:27.398862 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e690d2-8f32-412e-8749-db0699fe33eb None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:27.399139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e690d2-8f32-412e-8749-db0699fe33eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:27.417994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-93e690d2-8f32-412e-8749-db0699fe33eb None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:27.431936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-93e690d2-8f32-412e-8749-db0699fe33eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:27.431936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1002/2008] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:27.434730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e690d2-8f32-412e-8749-db0699fe33eb None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:27.443950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e690d2-8f32-412e-8749-db0699fe33eb None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:27.449935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9053e-d349-49b1-a03f-17a65f2171a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:27.454865 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9053e-d349-49b1-a03f-17a65f2171a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:27.454865 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9053e-d349-49b1-a03f-17a65f2171a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:27.454865 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9053e-d349-49b1-a03f-17a65f2171a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:27.454865 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9053e-d349-49b1-a03f-17a65f2171a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:27.454865 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9053e-d349-49b1-a03f-17a65f2171a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:27.454865 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9053e-d349-49b1-a03f-17a65f2171a0 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:27.456762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1007/2009] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:27 2022] POST /identity/v3/users => generated 306 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:27.460706 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6ab79-6ec0-455f-9009-00a4ebc9af68 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:27.464394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-84d6ab79-6ec0-455f-9009-00a4ebc9af68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:27.467042 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1003/2010] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:37:27.471950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9053e-d349-49b1-a03f-17a65f2171a0 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:27.480858 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6ab79-6ec0-455f-9009-00a4ebc9af68 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:27.483617 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9053e-d349-49b1-a03f-17a65f2171a0 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:27.485595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af1c83-bd3a-4d0e-83f4-fc97e623cf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:27.486207 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af1c83-bd3a-4d0e-83f4-fc97e623cf19 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:27.486490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af1c83-bd3a-4d0e-83f4-fc97e623cf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:27.486597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f304e2-0c1b-4dcf-b91d-5ff20397709d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:27.486766 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af1c83-bd3a-4d0e-83f4-fc97e623cf19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:27.487120 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f304e2-0c1b-4dcf-b91d-5ff20397709d admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:27.487403 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f304e2-0c1b-4dcf-b91d-5ff20397709d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:27.487508 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af1c83-bd3a-4d0e-83f4-fc97e623cf19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:27.487681 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f304e2-0c1b-4dcf-b91d-5ff20397709d admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:27.488313 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f304e2-0c1b-4dcf-b91d-5ff20397709d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:27.488973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af1c83-bd3a-4d0e-83f4-fc97e623cf19 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:27.490248 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f304e2-0c1b-4dcf-b91d-5ff20397709d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:27.499733 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1008/2011] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:27.503794 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af1c83-bd3a-4d0e-83f4-fc97e623cf19 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:27.514754 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af1c83-bd3a-4d0e-83f4-fc97e623cf19 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:27.520203 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec16c2b-ca35-4ca5-adae-d8dd6be84d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:27.520884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec16c2b-ca35-4ca5-adae-d8dd6be84d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:27.521161 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec16c2b-ca35-4ca5-adae-d8dd6be84d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:27.521443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec16c2b-ca35-4ca5-adae-d8dd6be84d89 admin admin] PATH_INFO: `/v3/projects/1c7253d01b034f7390e374132c671216/users/9d1a3b18bf614d639c2440a7a490ce0b/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:27.537293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec16c2b-ca35-4ca5-adae-d8dd6be84d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7253d01b034f7390e374132c671216, user_id=9d1a3b18bf614d639c2440a7a490ce0b, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:27.540321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec16c2b-ca35-4ca5-adae-d8dd6be84d89 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:27.617810 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1004/2012] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:27 2022] POST /identity/v3/projects => generated 379 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:27.627160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f304e2-0c1b-4dcf-b91d-5ff20397709d admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:27.628926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1009/2013] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:27 2022] PUT /identity/v3/projects/1c7253d01b034f7390e374132c671216/users/9d1a3b18bf614d639c2440a7a490ce0b/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:27.633142 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec16c2b-ca35-4ca5-adae-d8dd6be84d89 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:27.687677 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f304e2-0c1b-4dcf-b91d-5ff20397709d admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:27.693299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec16c2b-ca35-4ca5-adae-d8dd6be84d89 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:27.693474 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e1c98-0102-4659-a274-4f5a1adac3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:27.693943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e1c98-0102-4659-a274-4f5a1adac3fe admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:27.694222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e1c98-0102-4659-a274-4f5a1adac3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:27.694496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e1c98-0102-4659-a274-4f5a1adac3fe admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:27.695130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e1c98-0102-4659-a274-4f5a1adac3fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:27.697199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e1c98-0102-4659-a274-4f5a1adac3fe admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:27.698223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015617b-a03a-49fc-b6c3-8d87aeff4d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:27.698858 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015617b-a03a-49fc-b6c3-8d87aeff4d62 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:27.699137 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015617b-a03a-49fc-b6c3-8d87aeff4d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:27.699413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015617b-a03a-49fc-b6c3-8d87aeff4d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:27.700051 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015617b-a03a-49fc-b6c3-8d87aeff4d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:27.701800 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015617b-a03a-49fc-b6c3-8d87aeff4d62 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:27.711955 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1010/2014] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:27.715972 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015617b-a03a-49fc-b6c3-8d87aeff4d62 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:27.725328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015617b-a03a-49fc-b6c3-8d87aeff4d62 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:27.730914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598660ca-2f19-4938-844d-453fe549269e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:27.731534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598660ca-2f19-4938-844d-453fe549269e admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:27.731813 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598660ca-2f19-4938-844d-453fe549269e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:27.732092 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598660ca-2f19-4938-844d-453fe549269e admin admin] PATH_INFO: `/v3/projects/1c7253d01b034f7390e374132c671216/users/9d1a3b18bf614d639c2440a7a490ce0b/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:27.734036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598660ca-2f19-4938-844d-453fe549269e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7253d01b034f7390e374132c671216, user_id=9d1a3b18bf614d639c2440a7a490ce0b, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:27.736574 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598660ca-2f19-4938-844d-453fe549269e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:27.747065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1011/2015] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:27 2022] PUT /identity/v3/projects/1c7253d01b034f7390e374132c671216/users/9d1a3b18bf614d639c2440a7a490ce0b/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:27.757854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ec21c-e4f9-4b7e-9664-3a5c61869a01 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:27.758141 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ec21c-e4f9-4b7e-9664-3a5c61869a01 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:27.758419 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ec21c-e4f9-4b7e-9664-3a5c61869a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:27.772425 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1005/2016] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:27 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:37:27.777859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e1c98-0102-4659-a274-4f5a1adac3fe admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:27.785739 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f75ec21c-e4f9-4b7e-9664-3a5c61869a01 None None] MFA Rules not processed for user `9d1a3b18bf614d639c2440a7a490ce0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:27.786943 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f75ec21c-e4f9-4b7e-9664-3a5c61869a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:27.787581 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e1c98-0102-4659-a274-4f5a1adac3fe admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:27.840474 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:27.846346 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d548a2-eb0d-466e-8ffb-b59f0676cc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:27.849544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d548a2-eb0d-466e-8ffb-b59f0676cc27 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:27.849544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d548a2-eb0d-466e-8ffb-b59f0676cc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:27.849544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d548a2-eb0d-466e-8ffb-b59f0676cc27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:27.849544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d548a2-eb0d-466e-8ffb-b59f0676cc27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:27.850284 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d548a2-eb0d-466e-8ffb-b59f0676cc27 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:27.864276 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1006/2017] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:27.865403 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d548a2-eb0d-466e-8ffb-b59f0676cc27 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:27.866622 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1012/2018] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:27 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:27.875811 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d548a2-eb0d-466e-8ffb-b59f0676cc27 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:27.880514 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d61a7-2a96-4aee-b0a1-2c6a1be95653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:27.881398 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d61a7-2a96-4aee-b0a1-2c6a1be95653 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:27.881671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d61a7-2a96-4aee-b0a1-2c6a1be95653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:27.881944 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d61a7-2a96-4aee-b0a1-2c6a1be95653 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:27.882072 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ec21c-e4f9-4b7e-9664-3a5c61869a01 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:27.882576 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d61a7-2a96-4aee-b0a1-2c6a1be95653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:27.883370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f75ec21c-e4f9-4b7e-9664-3a5c61869a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:27.884351 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d61a7-2a96-4aee-b0a1-2c6a1be95653 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:27.895185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1007/2019] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:27.899507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d61a7-2a96-4aee-b0a1-2c6a1be95653 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:27.909717 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d61a7-2a96-4aee-b0a1-2c6a1be95653 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:27.914569 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f955a0-ab3e-4ee4-aebb-137a408af210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:27.923589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f955a0-ab3e-4ee4-aebb-137a408af210 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:27.923589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f955a0-ab3e-4ee4-aebb-137a408af210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:27.923589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f955a0-ab3e-4ee4-aebb-137a408af210 admin admin] PATH_INFO: `/v3/projects/65eb791d0bdf428fb9866a3ae2112072/users/8d7f8ae185a54830996f1181c3abea10/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:27.931606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f955a0-ab3e-4ee4-aebb-137a408af210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65eb791d0bdf428fb9866a3ae2112072, user_id=8d7f8ae185a54830996f1181c3abea10, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:27.934302 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f955a0-ab3e-4ee4-aebb-137a408af210 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:27.947132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ec21c-e4f9-4b7e-9664-3a5c61869a01 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:27.958557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899ffe2-16e1-447b-bb86-4dad20d62303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:27.958557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899ffe2-16e1-447b-bb86-4dad20d62303 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:27.958557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899ffe2-16e1-447b-bb86-4dad20d62303 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:27.958557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899ffe2-16e1-447b-bb86-4dad20d62303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:27.961473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899ffe2-16e1-447b-bb86-4dad20d62303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:27.963599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899ffe2-16e1-447b-bb86-4dad20d62303 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:27.972977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1013/2020] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:27 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:37:28.017003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1008/2021] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:27 2022] PUT /identity/v3/projects/65eb791d0bdf428fb9866a3ae2112072/users/8d7f8ae185a54830996f1181c3abea10/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:28.028018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d95fd3-1722-4dec-98ff-1c4ba563441a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:28.028367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d95fd3-1722-4dec-98ff-1c4ba563441a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:28.028587 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d95fd3-1722-4dec-98ff-1c4ba563441a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:28.059246 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-90d95fd3-1722-4dec-98ff-1c4ba563441a None None] MFA Rules not processed for user `8d7f8ae185a54830996f1181c3abea10`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:28.060518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-90d95fd3-1722-4dec-98ff-1c4ba563441a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:28.131715 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1014/2022] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:28.136160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f955a0-ab3e-4ee4-aebb-137a408af210 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:28.198526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:28.204769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f955a0-ab3e-4ee4-aebb-137a408af210 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:28.218067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60416f-2542-4399-9cc4-462f637c4c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:28.218067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60416f-2542-4399-9cc4-462f637c4c67 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:28.218067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60416f-2542-4399-9cc4-462f637c4c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:28.218067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60416f-2542-4399-9cc4-462f637c4c67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:28.218687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60416f-2542-4399-9cc4-462f637c4c67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:28.218687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60416f-2542-4399-9cc4-462f637c4c67 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:28.225527 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d95fd3-1722-4dec-98ff-1c4ba563441a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:28.226978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-90d95fd3-1722-4dec-98ff-1c4ba563441a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:28.292471 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d95fd3-1722-4dec-98ff-1c4ba563441a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:28.296361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f17f9-6863-4c4a-950a-0b57a5f5531e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:28.297036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f17f9-6863-4c4a-950a-0b57a5f5531e service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:28.297346 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f17f9-6863-4c4a-950a-0b57a5f5531e service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:28.297691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f17f9-6863-4c4a-950a-0b57a5f5531e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:28.300417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-fb2f17f9-6863-4c4a-950a-0b57a5f5531e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:28.314295 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1009/2023] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:28 2022] POST /identity/v3/projects => generated 379 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:28.318249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60416f-2542-4399-9cc4-462f637c4c67 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:28.363156 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f17f9-6863-4c4a-950a-0b57a5f5531e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:28.364596 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f17f9-6863-4c4a-950a-0b57a5f5531e service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:28.374048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1015/2024] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:28 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:37:28.385855 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60416f-2542-4399-9cc4-462f637c4c67 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:28.392896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93759c8f-b8cd-41e1-80e4-bc42f090286c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:28.393645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93759c8f-b8cd-41e1-80e4-bc42f090286c admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:28.393985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93759c8f-b8cd-41e1-80e4-bc42f090286c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:28.394266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93759c8f-b8cd-41e1-80e4-bc42f090286c admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:28.394909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93759c8f-b8cd-41e1-80e4-bc42f090286c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:28.397352 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93759c8f-b8cd-41e1-80e4-bc42f090286c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:28.525017 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1010/2025] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:28 2022] POST /identity/v3/users => generated 296 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:37:28.529045 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f17f9-6863-4c4a-950a-0b57a5f5531e service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:28.539664 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f17f9-6863-4c4a-950a-0b57a5f5531e service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:28.544670 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436daa4-d3f7-4196-b686-9103e994e147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:28.545319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436daa4-d3f7-4196-b686-9103e994e147 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:28.545612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436daa4-d3f7-4196-b686-9103e994e147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:28.545889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436daa4-d3f7-4196-b686-9103e994e147 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:28.546666 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436daa4-d3f7-4196-b686-9103e994e147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:28.548254 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436daa4-d3f7-4196-b686-9103e994e147 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:28.558781 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1016/2026] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:28.562967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93759c8f-b8cd-41e1-80e4-bc42f090286c admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:28.574804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93759c8f-b8cd-41e1-80e4-bc42f090286c admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:28.579799 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f368caf-4a13-496d-a591-772f22a928e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:28.580589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f368caf-4a13-496d-a591-772f22a928e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:28.580743 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f368caf-4a13-496d-a591-772f22a928e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:28.581020 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f368caf-4a13-496d-a591-772f22a928e3 admin admin] PATH_INFO: `/v3/projects/f78922046132460bac83daa204935413/users/570f888389774f18acbb33549108fab4/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:28.612398 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f368caf-4a13-496d-a591-772f22a928e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f78922046132460bac83daa204935413, user_id=570f888389774f18acbb33549108fab4, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:28.626866 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f368caf-4a13-496d-a591-772f22a928e3 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:28.691492 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1011/2027] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:28 2022] PUT /identity/v3/projects/f78922046132460bac83daa204935413/users/570f888389774f18acbb33549108fab4/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:28.695510 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436daa4-d3f7-4196-b686-9103e994e147 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:28.706561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436daa4-d3f7-4196-b686-9103e994e147 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:28.751513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:28.768542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfe6d7-e94a-4232-9b3b-7ea3ff6bbe0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:28.769338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfe6d7-e94a-4232-9b3b-7ea3ff6bbe0f admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:28.769618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfe6d7-e94a-4232-9b3b-7ea3ff6bbe0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:28.769892 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfe6d7-e94a-4232-9b3b-7ea3ff6bbe0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:28.770559 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cfe6d7-e94a-4232-9b3b-7ea3ff6bbe0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:28.772451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cfe6d7-e94a-4232-9b3b-7ea3ff6bbe0f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:28.787664 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1017/2028] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:28.787862 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f368caf-4a13-496d-a591-772f22a928e3 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:28.799060 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f368caf-4a13-496d-a591-772f22a928e3 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:28.803925 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f105b-3dae-4f28-b19c-a94ba8fee647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:28.804577 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f105b-3dae-4f28-b19c-a94ba8fee647 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:28.804860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f105b-3dae-4f28-b19c-a94ba8fee647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:28.805176 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f105b-3dae-4f28-b19c-a94ba8fee647 admin admin] PATH_INFO: `/v3/projects/f78922046132460bac83daa204935413/users/570f888389774f18acbb33549108fab4/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:28.807242 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f105b-3dae-4f28-b19c-a94ba8fee647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f78922046132460bac83daa204935413, user_id=570f888389774f18acbb33549108fab4, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:28.809926 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f105b-3dae-4f28-b19c-a94ba8fee647 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:28.838678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338229a-8ece-484a-aae4-4e948de412fb None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:28.838981 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338229a-8ece-484a-aae4-4e948de412fb None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:28.839254 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338229a-8ece-484a-aae4-4e948de412fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:28.841408 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1012/2029] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:28 2022] PUT /identity/v3/projects/f78922046132460bac83daa204935413/users/570f888389774f18acbb33549108fab4/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:28.869793 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-0338229a-8ece-484a-aae4-4e948de412fb None None] MFA Rules not processed for user `570f888389774f18acbb33549108fab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:28.871055 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0338229a-8ece-484a-aae4-4e948de412fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:28.937599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1018/2030] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:28.950837 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f105b-3dae-4f28-b19c-a94ba8fee647 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:29.009515 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f105b-3dae-4f28-b19c-a94ba8fee647 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:29.013236 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9324160-bcfd-4aa8-b308-1f7217ce13bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:29.013848 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9324160-bcfd-4aa8-b308-1f7217ce13bd service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:29.014126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9324160-bcfd-4aa8-b308-1f7217ce13bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:29.014400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9324160-bcfd-4aa8-b308-1f7217ce13bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:29.026072 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9324160-bcfd-4aa8-b308-1f7217ce13bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:29.027486 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9324160-bcfd-4aa8-b308-1f7217ce13bd service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:29.047350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1013/2031] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:29.058799 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:29.915223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338229a-8ece-484a-aae4-4e948de412fb None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:29.916799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0338229a-8ece-484a-aae4-4e948de412fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:29.982046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338229a-8ece-484a-aae4-4e948de412fb None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:29.987921 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36280ae2-f61e-4344-b18e-1689834de9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:29.988661 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36280ae2-f61e-4344-b18e-1689834de9b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:29.988964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36280ae2-f61e-4344-b18e-1689834de9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:29.989304 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36280ae2-f61e-4344-b18e-1689834de9b9 admin admin] PATH_INFO: `/v3/users/3ee15ca59e7545539898013ad4a4cffc` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:29.990434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36280ae2-f61e-4344-b18e-1689834de9b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ee15ca59e7545539898013ad4a4cffc)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:29.992375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36280ae2-f61e-4344-b18e-1689834de9b9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:30.170982 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:30.190607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-36280ae2-f61e-4344-b18e-1689834de9b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ee15ca59e7545539898013ad4a4cffc'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:30.191383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-36280ae2-f61e-4344-b18e-1689834de9b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ee15ca59e7545539898013ad4a4cffc'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:30.215973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9324160-bcfd-4aa8-b308-1f7217ce13bd service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:30.226961 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9324160-bcfd-4aa8-b308-1f7217ce13bd service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:30.231917 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4a624-2deb-4556-9ecd-b02a2aa42419 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '570f888389774f18acbb33549108fab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f78922046132460bac83daa204935413', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:30.233174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4a624-2deb-4556-9ecd-b02a2aa42419 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:30.237875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4a624-2deb-4556-9ecd-b02a2aa42419 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:30.237875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4a624-2deb-4556-9ecd-b02a2aa42419 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:30.237875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4a624-2deb-4556-9ecd-b02a2aa42419 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:30.237875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4a624-2deb-4556-9ecd-b02a2aa42419 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:30.237875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4a624-2deb-4556-9ecd-b02a2aa42419 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:30.255514 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1014/2032] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:37:30.263792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-36280ae2-f61e-4344-b18e-1689834de9b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ee15ca59e7545539898013ad4a4cffc'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:30.264503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-36280ae2-f61e-4344-b18e-1689834de9b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ee15ca59e7545539898013ad4a4cffc'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:30.264999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4a624-2deb-4556-9ecd-b02a2aa42419 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:30.267642 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-99b4a624-2deb-4556-9ecd-b02a2aa42419 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:30.279407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-36280ae2-f61e-4344-b18e-1689834de9b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ee15ca59e7545539898013ad4a4cffc'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:30.283366 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4a624-2deb-4556-9ecd-b02a2aa42419 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:30.289336 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becad085-3fea-4a5a-b3ec-286aec81d051 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '570f888389774f18acbb33549108fab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f78922046132460bac83daa204935413', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:30.291293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becad085-3fea-4a5a-b3ec-286aec81d051 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:30.291792 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becad085-3fea-4a5a-b3ec-286aec81d051 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:30.292238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becad085-3fea-4a5a-b3ec-286aec81d051 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:30.293044 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becad085-3fea-4a5a-b3ec-286aec81d051 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:30.295222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becad085-3fea-4a5a-b3ec-286aec81d051 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:30.295646 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1019/2033] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:29 2022] DELETE /identity/v3/users/3ee15ca59e7545539898013ad4a4cffc => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:30.300345 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67620ab-72df-4353-b9d7-10cd80babf69 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:30.300799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67620ab-72df-4353-b9d7-10cd80babf69 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:30.301627 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67620ab-72df-4353-b9d7-10cd80babf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:30.321226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c67620ab-72df-4353-b9d7-10cd80babf69 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:30.322337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c67620ab-72df-4353-b9d7-10cd80babf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:30.340184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:30.380266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1015/2034] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:30 2022] POST /identity/v3/projects => generated 354 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:30.384663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becad085-3fea-4a5a-b3ec-286aec81d051 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:30.389903 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1020/2035] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:30.406134 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67620ab-72df-4353-b9d7-10cd80babf69 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:30.407527 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c67620ab-72df-4353-b9d7-10cd80babf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:30.469940 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becad085-3fea-4a5a-b3ec-286aec81d051 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:30.476029 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def6a045-35c8-4a86-a3e1-b90b45482f2d tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '570f888389774f18acbb33549108fab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f78922046132460bac83daa204935413', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:30.476641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6a045-35c8-4a86-a3e1-b90b45482f2d tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:30.476922 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6a045-35c8-4a86-a3e1-b90b45482f2d tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:30.477284 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6a045-35c8-4a86-a3e1-b90b45482f2d tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:30.478619 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def6a045-35c8-4a86-a3e1-b90b45482f2d tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:30.478914 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def6a045-35c8-4a86-a3e1-b90b45482f2d tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:30.480974 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def6a045-35c8-4a86-a3e1-b90b45482f2d tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:30.482349 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67620ab-72df-4353-b9d7-10cd80babf69 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:30.494300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1016/2036] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:37:30.541544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce07d96-7373-4b57-acb0-a9b92c158c41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:30.542180 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce07d96-7373-4b57-acb0-a9b92c158c41 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:30.542446 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce07d96-7373-4b57-acb0-a9b92c158c41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:30.542817 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce07d96-7373-4b57-acb0-a9b92c158c41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:30.544103 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def6a045-35c8-4a86-a3e1-b90b45482f2d tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:30.553922 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def6a045-35c8-4a86-a3e1-b90b45482f2d tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:30.559547 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd568b-3617-4606-9a74-ccd08e3564e1 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '570f888389774f18acbb33549108fab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f78922046132460bac83daa204935413', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:30.560203 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd568b-3617-4606-9a74-ccd08e3564e1 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:30.560486 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd568b-3617-4606-9a74-ccd08e3564e1 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:30.560771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd568b-3617-4606-9a74-ccd08e3564e1 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] PATH_INFO: `/v3/projects/8b97062b9bf04b769bb07a63d4ad1bc7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:30.561914 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd568b-3617-4606-9a74-ccd08e3564e1 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8b97062b9bf04b769bb07a63d4ad1bc7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:30.563825 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd568b-3617-4606-9a74-ccd08e3564e1 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:30.610228 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce07d96-7373-4b57-acb0-a9b92c158c41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:30.611575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce07d96-7373-4b57-acb0-a9b92c158c41 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:30.622068 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1021/2037] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:37:30.700655 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-73cd568b-3617-4606-9a74-ccd08e3564e1 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b97062b9bf04b769bb07a63d4ad1bc7', 'request_id': 'req-73cd568b-3617-4606-9a74-ccd08e3564e1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:30.706612 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-73cd568b-3617-4606-9a74-ccd08e3564e1 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b97062b9bf04b769bb07a63d4ad1bc7', 'request_id': 'req-73cd568b-3617-4606-9a74-ccd08e3564e1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:30.717547 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-73cd568b-3617-4606-9a74-ccd08e3564e1 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b97062b9bf04b769bb07a63d4ad1bc7', 'request_id': 'req-73cd568b-3617-4606-9a74-ccd08e3564e1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:30.850264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-73cd568b-3617-4606-9a74-ccd08e3564e1 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] The token cache is being invalidate because project 8b97062b9bf04b769bb07a63d4ad1bc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:30.850264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-73cd568b-3617-4606-9a74-ccd08e3564e1 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:30.857510 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-73cd568b-3617-4606-9a74-ccd08e3564e1 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:30.859646 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1017/2038] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:30 2022] DELETE /identity/v3/projects/8b97062b9bf04b769bb07a63d4ad1bc7 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:31.125365 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce07d96-7373-4b57-acb0-a9b92c158c41 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:31.125365 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-fce07d96-7373-4b57-acb0-a9b92c158c41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:31.133130 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce07d96-7373-4b57-acb0-a9b92c158c41 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:31.193995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695eff8-ab2b-4ee1-a6b7-b3a3c2feb54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:31.194623 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695eff8-ab2b-4ee1-a6b7-b3a3c2feb54a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:31.194898 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695eff8-ab2b-4ee1-a6b7-b3a3c2feb54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:31.195190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695eff8-ab2b-4ee1-a6b7-b3a3c2feb54a admin admin] PATH_INFO: `/v3/projects/b411a2825f6542bab753a581c78fe9ed` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:31.196283 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695eff8-ab2b-4ee1-a6b7-b3a3c2feb54a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b411a2825f6542bab753a581c78fe9ed)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:31.198213 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695eff8-ab2b-4ee1-a6b7-b3a3c2feb54a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:31.305192 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b695eff8-ab2b-4ee1-a6b7-b3a3c2feb54a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b411a2825f6542bab753a581c78fe9ed', 'request_id': 'req-b695eff8-ab2b-4ee1-a6b7-b3a3c2feb54a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:31.311203 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b695eff8-ab2b-4ee1-a6b7-b3a3c2feb54a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b411a2825f6542bab753a581c78fe9ed', 'request_id': 'req-b695eff8-ab2b-4ee1-a6b7-b3a3c2feb54a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:31.317406 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b695eff8-ab2b-4ee1-a6b7-b3a3c2feb54a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b411a2825f6542bab753a581c78fe9ed', 'request_id': 'req-b695eff8-ab2b-4ee1-a6b7-b3a3c2feb54a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:31.349968 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:31.406017 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b695eff8-ab2b-4ee1-a6b7-b3a3c2feb54a admin admin] The token cache is being invalidate because project b411a2825f6542bab753a581c78fe9ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:31.406320 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b695eff8-ab2b-4ee1-a6b7-b3a3c2feb54a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:31.407020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b695eff8-ab2b-4ee1-a6b7-b3a3c2feb54a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:31.408512 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1022/2039] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:31 2022] DELETE /identity/v3/projects/b411a2825f6542bab753a581c78fe9ed => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:31.412574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd568b-3617-4606-9a74-ccd08e3564e1 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:31.414063 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-73cd568b-3617-4606-9a74-ccd08e3564e1 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:31.427580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd568b-3617-4606-9a74-ccd08e3564e1 tempest-VolumeQuotaClassesTest-664411345 tempest-VolumeQuotaClassesTest-664411345-project-admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:31.484600 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e95fbc-ad73-4cc8-854c-d3234d80890d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:31.485263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e95fbc-ad73-4cc8-854c-d3234d80890d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:31.485606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e95fbc-ad73-4cc8-854c-d3234d80890d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:31.485860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e95fbc-ad73-4cc8-854c-d3234d80890d admin admin] PATH_INFO: `/v3/users/d7c9bd0efc0e4748964d14ac9b6e281e` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:31.486972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e95fbc-ad73-4cc8-854c-d3234d80890d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7c9bd0efc0e4748964d14ac9b6e281e)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:31.488932 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e95fbc-ad73-4cc8-854c-d3234d80890d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:31.556102 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695eff8-ab2b-4ee1-a6b7-b3a3c2feb54a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:31.557563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b695eff8-ab2b-4ee1-a6b7-b3a3c2feb54a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:31.570279 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695eff8-ab2b-4ee1-a6b7-b3a3c2feb54a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:31.575463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede670b2-6801-4f4f-a439-95c14563e5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:31.576085 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede670b2-6801-4f4f-a439-95c14563e5e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:31.577742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede670b2-6801-4f4f-a439-95c14563e5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:31.577742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede670b2-6801-4f4f-a439-95c14563e5e7 admin admin] PATH_INFO: `/v3/users/8d7f8ae185a54830996f1181c3abea10` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:31.577937 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede670b2-6801-4f4f-a439-95c14563e5e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d7f8ae185a54830996f1181c3abea10)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:31.579830 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede670b2-6801-4f4f-a439-95c14563e5e7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:31.695438 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-34e95fbc-ad73-4cc8-854c-d3234d80890d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7c9bd0efc0e4748964d14ac9b6e281e'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:31.744736 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ede670b2-6801-4f4f-a439-95c14563e5e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d7f8ae185a54830996f1181c3abea10'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:31.745415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ede670b2-6801-4f4f-a439-95c14563e5e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d7f8ae185a54830996f1181c3abea10'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:31.753139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-34e95fbc-ad73-4cc8-854c-d3234d80890d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7c9bd0efc0e4748964d14ac9b6e281e'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:31.763658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-34e95fbc-ad73-4cc8-854c-d3234d80890d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7c9bd0efc0e4748964d14ac9b6e281e'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:31.763658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-34e95fbc-ad73-4cc8-854c-d3234d80890d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7c9bd0efc0e4748964d14ac9b6e281e'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:31.763658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-34e95fbc-ad73-4cc8-854c-d3234d80890d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7c9bd0efc0e4748964d14ac9b6e281e'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:31.781607 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1018/2040] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:31 2022] DELETE /identity/v3/users/d7c9bd0efc0e4748964d14ac9b6e281e => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:31.845141 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ede670b2-6801-4f4f-a439-95c14563e5e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d7f8ae185a54830996f1181c3abea10'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:31.846121 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ede670b2-6801-4f4f-a439-95c14563e5e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d7f8ae185a54830996f1181c3abea10'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:31.858612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ede670b2-6801-4f4f-a439-95c14563e5e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d7f8ae185a54830996f1181c3abea10'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:31.862539 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:31.871986 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1023/2041] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:31 2022] DELETE /identity/v3/users/8d7f8ae185a54830996f1181c3abea10 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:31.876497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93884d04-8649-41c7-ba62-7f5339357adf None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:31.876838 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93884d04-8649-41c7-ba62-7f5339357adf None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:31.877269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93884d04-8649-41c7-ba62-7f5339357adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:31.897397 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-93884d04-8649-41c7-ba62-7f5339357adf None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:31.898608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-93884d04-8649-41c7-ba62-7f5339357adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:31.958374 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1019/2042] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:31.968979 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede670b2-6801-4f4f-a439-95c14563e5e7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:31.970621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ede670b2-6801-4f4f-a439-95c14563e5e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:32.057167 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede670b2-6801-4f4f-a439-95c14563e5e7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:32.076942 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e5201-8512-49d9-9581-1cfdf4c1ffa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:32.080269 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e5201-8512-49d9-9581-1cfdf4c1ffa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:32.080269 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e5201-8512-49d9-9581-1cfdf4c1ffa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:32.080269 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e5201-8512-49d9-9581-1cfdf4c1ffa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:32.096870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e5201-8512-49d9-9581-1cfdf4c1ffa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:32.098602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e5201-8512-49d9-9581-1cfdf4c1ffa2 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:32.126550 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1024/2043] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:37:32.241833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93884d04-8649-41c7-ba62-7f5339357adf None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:32.244071 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-93884d04-8649-41c7-ba62-7f5339357adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:32.262340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93884d04-8649-41c7-ba62-7f5339357adf None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:32.272644 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ed738-26c5-4952-b987-72632fa270ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:32.273826 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ed738-26c5-4952-b987-72632fa270ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:32.274224 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ed738-26c5-4952-b987-72632fa270ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:32.274566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ed738-26c5-4952-b987-72632fa270ce admin admin] PATH_INFO: `/v3/projects/9f83fa9c99824fdd81bc9eb8b2d2adfb` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:32.275848 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ed738-26c5-4952-b987-72632fa270ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f83fa9c99824fdd81bc9eb8b2d2adfb)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:32.278537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ed738-26c5-4952-b987-72632fa270ce admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:32.399633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-371ed738-26c5-4952-b987-72632fa270ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f83fa9c99824fdd81bc9eb8b2d2adfb', 'request_id': 'req-371ed738-26c5-4952-b987-72632fa270ce'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:32.406519 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-371ed738-26c5-4952-b987-72632fa270ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f83fa9c99824fdd81bc9eb8b2d2adfb', 'request_id': 'req-371ed738-26c5-4952-b987-72632fa270ce'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:32.415370 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-371ed738-26c5-4952-b987-72632fa270ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f83fa9c99824fdd81bc9eb8b2d2adfb', 'request_id': 'req-371ed738-26c5-4952-b987-72632fa270ce'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:32.490209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-371ed738-26c5-4952-b987-72632fa270ce admin admin] The token cache is being invalidate because project 9f83fa9c99824fdd81bc9eb8b2d2adfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:32.490492 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-371ed738-26c5-4952-b987-72632fa270ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:32.493998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-371ed738-26c5-4952-b987-72632fa270ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:32.493998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1020/2044] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:32 2022] DELETE /identity/v3/projects/9f83fa9c99824fdd81bc9eb8b2d2adfb => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:32.498572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d24dc4-1592-4801-bf7b-245e648ec15b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:32.499051 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d24dc4-1592-4801-bf7b-245e648ec15b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:32.499246 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d24dc4-1592-4801-bf7b-245e648ec15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:32.520170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-89d24dc4-1592-4801-bf7b-245e648ec15b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:32.521461 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-89d24dc4-1592-4801-bf7b-245e648ec15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:32.583355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1025/2045] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:32.592377 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529af2a-7f5e-46a7-8d25-46480e30ebdd None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:32.592658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529af2a-7f5e-46a7-8d25-46480e30ebdd None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:32.592933 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529af2a-7f5e-46a7-8d25-46480e30ebdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:32.613779 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-4529af2a-7f5e-46a7-8d25-46480e30ebdd None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:32.614981 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4529af2a-7f5e-46a7-8d25-46480e30ebdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:32.622864 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1021/2046] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:32.627170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d24dc4-1592-4801-bf7b-245e648ec15b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:32.637107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d24dc4-1592-4801-bf7b-245e648ec15b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:32.642749 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454fb87-628d-4ed0-9baf-cc6c70dd6ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:32.646220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454fb87-628d-4ed0-9baf-cc6c70dd6ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:32.646220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454fb87-628d-4ed0-9baf-cc6c70dd6ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:32.646220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454fb87-628d-4ed0-9baf-cc6c70dd6ce8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:32.646220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454fb87-628d-4ed0-9baf-cc6c70dd6ce8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:32.646220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454fb87-628d-4ed0-9baf-cc6c70dd6ce8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:32.646663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529af2a-7f5e-46a7-8d25-46480e30ebdd None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:32.646747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454fb87-628d-4ed0-9baf-cc6c70dd6ce8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:32.647113 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4529af2a-7f5e-46a7-8d25-46480e30ebdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:32.662134 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529af2a-7f5e-46a7-8d25-46480e30ebdd None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:32.668782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1026/2047] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:37:32.673993 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454fb87-628d-4ed0-9baf-cc6c70dd6ce8 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:32.675607 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c1670-8afa-4c2c-bc0c-cb0c0c81d52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:32.676263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c1670-8afa-4c2c-bc0c-cb0c0c81d52c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:32.680539 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c1670-8afa-4c2c-bc0c-cb0c0c81d52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:32.680539 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c1670-8afa-4c2c-bc0c-cb0c0c81d52c admin admin] PATH_INFO: `/v3/projects/65eb791d0bdf428fb9866a3ae2112072` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:32.680539 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c1670-8afa-4c2c-bc0c-cb0c0c81d52c admin admin] RBAC: Authorizing `identity:delete_project(project_id=65eb791d0bdf428fb9866a3ae2112072)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:32.680539 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c1670-8afa-4c2c-bc0c-cb0c0c81d52c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:32.689710 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454fb87-628d-4ed0-9baf-cc6c70dd6ce8 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:32.698061 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e91e0a-351f-462d-90fc-2776c5c953f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:32.698712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e91e0a-351f-462d-90fc-2776c5c953f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:32.698987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e91e0a-351f-462d-90fc-2776c5c953f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:32.699257 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e91e0a-351f-462d-90fc-2776c5c953f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:32.699939 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e91e0a-351f-462d-90fc-2776c5c953f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:32.701866 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e91e0a-351f-462d-90fc-2776c5c953f4 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:32.704744 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:32.736120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:32.810544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-d91c1670-8afa-4c2c-bc0c-cb0c0c81d52c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65eb791d0bdf428fb9866a3ae2112072', 'request_id': 'req-d91c1670-8afa-4c2c-bc0c-cb0c0c81d52c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:32.816259 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-d91c1670-8afa-4c2c-bc0c-cb0c0c81d52c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65eb791d0bdf428fb9866a3ae2112072', 'request_id': 'req-d91c1670-8afa-4c2c-bc0c-cb0c0c81d52c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:32.823644 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-d91c1670-8afa-4c2c-bc0c-cb0c0c81d52c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65eb791d0bdf428fb9866a3ae2112072', 'request_id': 'req-d91c1670-8afa-4c2c-bc0c-cb0c0c81d52c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:32.835706 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1027/2048] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:32 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:37:32.840383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e91e0a-351f-462d-90fc-2776c5c953f4 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:32.903754 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e91e0a-351f-462d-90fc-2776c5c953f4 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:32.908725 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cc8d0-c800-4e23-8771-18bdd6b05e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:32.909360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cc8d0-c800-4e23-8771-18bdd6b05e5d admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:32.909634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cc8d0-c800-4e23-8771-18bdd6b05e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:32.909903 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cc8d0-c800-4e23-8771-18bdd6b05e5d admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:32.910532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22cc8d0-c800-4e23-8771-18bdd6b05e5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:32.912479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22cc8d0-c800-4e23-8771-18bdd6b05e5d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:32.926871 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-d91c1670-8afa-4c2c-bc0c-cb0c0c81d52c admin admin] The token cache is being invalidate because project 65eb791d0bdf428fb9866a3ae2112072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:32.931517 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-d91c1670-8afa-4c2c-bc0c-cb0c0c81d52c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:32.931517 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-d91c1670-8afa-4c2c-bc0c-cb0c0c81d52c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:32.931517 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1022/2049] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:32 2022] DELETE /identity/v3/projects/65eb791d0bdf428fb9866a3ae2112072 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:32.933196 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c1670-8afa-4c2c-bc0c-cb0c0c81d52c admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:32.934512 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d91c1670-8afa-4c2c-bc0c-cb0c0c81d52c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:32.998857 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c1670-8afa-4c2c-bc0c-cb0c0c81d52c admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:33.004202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37d682-412d-4bf1-8220-250aec58710a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:33.004880 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37d682-412d-4bf1-8220-250aec58710a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:33.005016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1028/2050] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:32 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:37:33.008982 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37d682-412d-4bf1-8220-250aec58710a admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:33.008982 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37d682-412d-4bf1-8220-250aec58710a admin admin] PATH_INFO: `/v3/users/570f888389774f18acbb33549108fab4` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:33.008982 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec37d682-412d-4bf1-8220-250aec58710a admin admin] RBAC: Authorizing `identity:delete_user(user_id=570f888389774f18acbb33549108fab4)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:33.009486 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec37d682-412d-4bf1-8220-250aec58710a admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:33.012390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cc8d0-c800-4e23-8771-18bdd6b05e5d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:33.014154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e22cc8d0-c800-4e23-8771-18bdd6b05e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:33.027203 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cc8d0-c800-4e23-8771-18bdd6b05e5d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:33.032754 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b0cdf-991e-49ed-b925-c8668533a1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:33.033392 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b0cdf-991e-49ed-b925-c8668533a1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:33.033695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b0cdf-991e-49ed-b925-c8668533a1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:33.033974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b0cdf-991e-49ed-b925-c8668533a1d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:33.034646 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b0cdf-991e-49ed-b925-c8668533a1d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:33.036276 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b0cdf-991e-49ed-b925-c8668533a1d7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:33.046341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1029/2051] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:33.050371 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b0cdf-991e-49ed-b925-c8668533a1d7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:33.059722 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b0cdf-991e-49ed-b925-c8668533a1d7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:33.064460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72639c5a-3266-445c-85ed-4d7a6ee8c001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:33.065025 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72639c5a-3266-445c-85ed-4d7a6ee8c001 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:33.065334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72639c5a-3266-445c-85ed-4d7a6ee8c001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:33.065609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72639c5a-3266-445c-85ed-4d7a6ee8c001 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:33.066267 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72639c5a-3266-445c-85ed-4d7a6ee8c001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:33.067820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72639c5a-3266-445c-85ed-4d7a6ee8c001 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:33.079763 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1030/2052] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:33.084023 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72639c5a-3266-445c-85ed-4d7a6ee8c001 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:33.095267 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72639c5a-3266-445c-85ed-4d7a6ee8c001 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:33.100069 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47da5f-dbc5-405d-a857-7a4b4320244a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:33.100729 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47da5f-dbc5-405d-a857-7a4b4320244a admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:33.101006 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47da5f-dbc5-405d-a857-7a4b4320244a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:33.101311 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47da5f-dbc5-405d-a857-7a4b4320244a admin admin] PATH_INFO: `/v3/projects/5765abd4468441e790c614f3982279cc/users/cdf6ac2a995f40cfaa24385f514e8e63/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:33.116908 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47da5f-dbc5-405d-a857-7a4b4320244a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5765abd4468441e790c614f3982279cc, user_id=cdf6ac2a995f40cfaa24385f514e8e63, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:33.120239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47da5f-dbc5-405d-a857-7a4b4320244a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:33.210196 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ec37d682-412d-4bf1-8220-250aec58710a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '570f888389774f18acbb33549108fab4'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:33.230533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1031/2053] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:33 2022] PUT /identity/v3/projects/5765abd4468441e790c614f3982279cc/users/cdf6ac2a995f40cfaa24385f514e8e63/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:33.243337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ce36d-9da0-48c6-ace9-944bb0650f94 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:33.243646 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ce36d-9da0-48c6-ace9-944bb0650f94 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:33.243940 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ce36d-9da0-48c6-ace9-944bb0650f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:33.275346 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-565ce36d-9da0-48c6-ace9-944bb0650f94 None None] MFA Rules not processed for user `cdf6ac2a995f40cfaa24385f514e8e63`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:33.276791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-565ce36d-9da0-48c6-ace9-944bb0650f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:33.309815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ec37d682-412d-4bf1-8220-250aec58710a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '570f888389774f18acbb33549108fab4'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:33.310451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ec37d682-412d-4bf1-8220-250aec58710a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '570f888389774f18acbb33549108fab4'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:33.311068 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ec37d682-412d-4bf1-8220-250aec58710a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '570f888389774f18acbb33549108fab4'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:33.318609 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ec37d682-412d-4bf1-8220-250aec58710a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '570f888389774f18acbb33549108fab4'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:33.338210 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1023/2054] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:32 2022] DELETE /identity/v3/users/570f888389774f18acbb33549108fab4 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:33.349508 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1032/2055] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:33.363419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37d682-412d-4bf1-8220-250aec58710a admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:33.364940 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-ec37d682-412d-4bf1-8220-250aec58710a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:33.438701 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37d682-412d-4bf1-8220-250aec58710a admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:33.446838 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a967bf-ad0b-4a34-aa20-3217e8a1e4d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:33.460302 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a967bf-ad0b-4a34-aa20-3217e8a1e4d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:33.465626 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a967bf-ad0b-4a34-aa20-3217e8a1e4d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:33.465626 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a967bf-ad0b-4a34-aa20-3217e8a1e4d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:33.531314 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a967bf-ad0b-4a34-aa20-3217e8a1e4d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:33.532685 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a967bf-ad0b-4a34-aa20-3217e8a1e4d7 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:33.542612 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1024/2056] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:33.568752 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ce36d-9da0-48c6-ace9-944bb0650f94 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:33.571518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-565ce36d-9da0-48c6-ace9-944bb0650f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:33.576329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:33.676418 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ce36d-9da0-48c6-ace9-944bb0650f94 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:33.761923 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209106fa-b68b-455f-a6d8-729189176da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:33.762699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209106fa-b68b-455f-a6d8-729189176da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:33.763037 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209106fa-b68b-455f-a6d8-729189176da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:33.763514 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209106fa-b68b-455f-a6d8-729189176da8 admin admin] PATH_INFO: `/v3/users/cdf6ac2a995f40cfaa24385f514e8e63` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:33.764893 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209106fa-b68b-455f-a6d8-729189176da8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdf6ac2a995f40cfaa24385f514e8e63)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:33.767253 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209106fa-b68b-455f-a6d8-729189176da8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:33.834107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a967bf-ad0b-4a34-aa20-3217e8a1e4d7 service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:33.835530 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-50a967bf-ad0b-4a34-aa20-3217e8a1e4d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:33.855308 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a967bf-ad0b-4a34-aa20-3217e8a1e4d7 service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:33.861913 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edee73-b920-479a-9812-e56a36977296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:33.862555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edee73-b920-479a-9812-e56a36977296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:33.862835 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edee73-b920-479a-9812-e56a36977296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:33.863110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edee73-b920-479a-9812-e56a36977296 admin admin] PATH_INFO: `/v3/users/c9fa148051334dc9a8c781f661225ba4` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:33.864254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edee73-b920-479a-9812-e56a36977296 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9fa148051334dc9a8c781f661225ba4)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:33.866372 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edee73-b920-479a-9812-e56a36977296 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:33.981902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-209106fa-b68b-455f-a6d8-729189176da8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdf6ac2a995f40cfaa24385f514e8e63'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:33.982561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-209106fa-b68b-455f-a6d8-729189176da8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdf6ac2a995f40cfaa24385f514e8e63'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:34.047203 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-209106fa-b68b-455f-a6d8-729189176da8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdf6ac2a995f40cfaa24385f514e8e63'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:34.047810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-209106fa-b68b-455f-a6d8-729189176da8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdf6ac2a995f40cfaa24385f514e8e63'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:34.061790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-209106fa-b68b-455f-a6d8-729189176da8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdf6ac2a995f40cfaa24385f514e8e63'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:34.073966 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1033/2057] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:33 2022] DELETE /identity/v3/users/cdf6ac2a995f40cfaa24385f514e8e63 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:34.074792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209106fa-b68b-455f-a6d8-729189176da8 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:34.076154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-209106fa-b68b-455f-a6d8-729189176da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:34.124928 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-a7edee73-b920-479a-9812-e56a36977296 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9fa148051334dc9a8c781f661225ba4'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:34.143416 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209106fa-b68b-455f-a6d8-729189176da8 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:34.149544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d3a3c-76ce-495a-8d17-b071d3deb015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:34.150189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d3a3c-76ce-495a-8d17-b071d3deb015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:34.150472 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d3a3c-76ce-495a-8d17-b071d3deb015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:34.150747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d3a3c-76ce-495a-8d17-b071d3deb015 admin admin] PATH_INFO: `/v3/projects/f78922046132460bac83daa204935413` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:34.152096 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d3a3c-76ce-495a-8d17-b071d3deb015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f78922046132460bac83daa204935413)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:34.154035 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d3a3c-76ce-495a-8d17-b071d3deb015 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:34.164434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:DELETE=26,keystone:SELECT=71,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:34.217546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-a7edee73-b920-479a-9812-e56a36977296 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9fa148051334dc9a8c781f661225ba4'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:34.218307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-a7edee73-b920-479a-9812-e56a36977296 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9fa148051334dc9a8c781f661225ba4'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:34.219028 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-a7edee73-b920-479a-9812-e56a36977296 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9fa148051334dc9a8c781f661225ba4'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:34.230608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-a7edee73-b920-479a-9812-e56a36977296 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9fa148051334dc9a8c781f661225ba4'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:34.255264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1025/2058] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:33 2022] DELETE /identity/v3/users/c9fa148051334dc9a8c781f661225ba4 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:34.271849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ecba9-af1f-45ca-98d7-ffccad7a6211 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:34.271849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ecba9-af1f-45ca-98d7-ffccad7a6211 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:34.271849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ecba9-af1f-45ca-98d7-ffccad7a6211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:34.285765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-165ecba9-af1f-45ca-98d7-ffccad7a6211 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:34.287413 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-165ecba9-af1f-45ca-98d7-ffccad7a6211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:34.317881 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d75d3a3c-76ce-495a-8d17-b071d3deb015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f78922046132460bac83daa204935413', 'request_id': 'req-d75d3a3c-76ce-495a-8d17-b071d3deb015'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:34.324623 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d75d3a3c-76ce-495a-8d17-b071d3deb015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f78922046132460bac83daa204935413', 'request_id': 'req-d75d3a3c-76ce-495a-8d17-b071d3deb015'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:34.331202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d75d3a3c-76ce-495a-8d17-b071d3deb015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f78922046132460bac83daa204935413', 'request_id': 'req-d75d3a3c-76ce-495a-8d17-b071d3deb015'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:34.362888 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1026/2059] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:34.369357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba49b6-6d11-4637-b179-798fcd767994 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:34.370829 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba49b6-6d11-4637-b179-798fcd767994 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:34.371146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba49b6-6d11-4637-b179-798fcd767994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:34.400088 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-73ba49b6-6d11-4637-b179-798fcd767994 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:34.400088 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-73ba49b6-6d11-4637-b179-798fcd767994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:34.466113 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1027/2060] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:34.466113 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba49b6-6d11-4637-b179-798fcd767994 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:34.467303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d75d3a3c-76ce-495a-8d17-b071d3deb015 admin admin] The token cache is being invalidate because project f78922046132460bac83daa204935413 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:34.467303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d75d3a3c-76ce-495a-8d17-b071d3deb015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:34.467467 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-73ba49b6-6d11-4637-b179-798fcd767994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:34.468076 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-d75d3a3c-76ce-495a-8d17-b071d3deb015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:34.470123 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1034/2061] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:34 2022] DELETE /identity/v3/projects/f78922046132460bac83daa204935413 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:34.475199 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292883d-f303-4c79-bd2c-87d687de4e4e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:34.475562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292883d-f303-4c79-bd2c-87d687de4e4e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:34.475913 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292883d-f303-4c79-bd2c-87d687de4e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:34.501146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6292883d-f303-4c79-bd2c-87d687de4e4e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:34.502589 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6292883d-f303-4c79-bd2c-87d687de4e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:34.520465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1035/2062] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:34.520465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292883d-f303-4c79-bd2c-87d687de4e4e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:34.520465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6292883d-f303-4c79-bd2c-87d687de4e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:34.553212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba49b6-6d11-4637-b179-798fcd767994 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:34.557876 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aeb18e-5ebd-4940-a642-c188368b723b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:34.558497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aeb18e-5ebd-4940-a642-c188368b723b service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:34.558779 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aeb18e-5ebd-4940-a642-c188368b723b service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:34.559823 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aeb18e-5ebd-4940-a642-c188368b723b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:34.561911 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:34.564331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-47aeb18e-5ebd-4940-a642-c188368b723b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:34.577200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aeb18e-5ebd-4940-a642-c188368b723b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:34.578647 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aeb18e-5ebd-4940-a642-c188368b723b service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:34.603257 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1028/2063] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:34.604594 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abdb1d-ac11-4fb2-a6bf-8ead1de0f442 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:34.604986 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abdb1d-ac11-4fb2-a6bf-8ead1de0f442 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:34.605354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abdb1d-ac11-4fb2-a6bf-8ead1de0f442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:34.609792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292883d-f303-4c79-bd2c-87d687de4e4e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:34.613949 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a35966-bbb2-43f9-a85e-4e0dd71fa403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:34.614665 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a35966-bbb2-43f9-a85e-4e0dd71fa403 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:34.614946 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a35966-bbb2-43f9-a85e-4e0dd71fa403 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:34.615226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a35966-bbb2-43f9-a85e-4e0dd71fa403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:34.617045 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-53a35966-bbb2-43f9-a85e-4e0dd71fa403 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:34.625817 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-b5abdb1d-ac11-4fb2-a6bf-8ead1de0f442 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:34.627156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b5abdb1d-ac11-4fb2-a6bf-8ead1de0f442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:34.629577 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a35966-bbb2-43f9-a85e-4e0dd71fa403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:34.631229 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a35966-bbb2-43f9-a85e-4e0dd71fa403 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:34.639082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1029/2064] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:34.642047 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5abdb1d-ac11-4fb2-a6bf-8ead1de0f442 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:34.648158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1036/2065] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:34.651940 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5abdb1d-ac11-4fb2-a6bf-8ead1de0f442 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:34.656980 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf209c8f-3d2b-45a7-985b-5c14dfe29daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:34.662727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf209c8f-3d2b-45a7-985b-5c14dfe29daa admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:34.662727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf209c8f-3d2b-45a7-985b-5c14dfe29daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:34.662727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf209c8f-3d2b-45a7-985b-5c14dfe29daa admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:34.662727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf209c8f-3d2b-45a7-985b-5c14dfe29daa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:34.662727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf209c8f-3d2b-45a7-985b-5c14dfe29daa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:34.662727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf209c8f-3d2b-45a7-985b-5c14dfe29daa admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:34.676693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1030/2066] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:37:34.677919 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a35966-bbb2-43f9-a85e-4e0dd71fa403 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:34.688571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a35966-bbb2-43f9-a85e-4e0dd71fa403 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:34.693388 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d91885-084f-4219-9fef-40efdaf856fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:34.693996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d91885-084f-4219-9fef-40efdaf856fd admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:34.694543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d91885-084f-4219-9fef-40efdaf856fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:34.694824 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d91885-084f-4219-9fef-40efdaf856fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:34.695464 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d91885-084f-4219-9fef-40efdaf856fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:34.698057 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d91885-084f-4219-9fef-40efdaf856fd admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:34.816867 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1037/2067] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:34 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:34.818264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf209c8f-3d2b-45a7-985b-5c14dfe29daa admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:34.829221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf209c8f-3d2b-45a7-985b-5c14dfe29daa admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:34.913189 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c464ce-834b-429e-9fa8-f7b67d6cde39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:34.914157 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c464ce-834b-429e-9fa8-f7b67d6cde39 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:34.914516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c464ce-834b-429e-9fa8-f7b67d6cde39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:34.914859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c464ce-834b-429e-9fa8-f7b67d6cde39 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:34.915608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c464ce-834b-429e-9fa8-f7b67d6cde39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:34.917811 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c464ce-834b-429e-9fa8-f7b67d6cde39 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:35.026562 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1031/2068] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:34 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:35.028512 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d91885-084f-4219-9fef-40efdaf856fd admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:35.039268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d91885-084f-4219-9fef-40efdaf856fd admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:35.044195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208169e0-fae6-4892-ad5f-e025c80db804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:35.044790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208169e0-fae6-4892-ad5f-e025c80db804 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:35.045082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208169e0-fae6-4892-ad5f-e025c80db804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:35.046126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208169e0-fae6-4892-ad5f-e025c80db804 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:35.046126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208169e0-fae6-4892-ad5f-e025c80db804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:35.047591 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208169e0-fae6-4892-ad5f-e025c80db804 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:35.058486 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1038/2069] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:35.062480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c464ce-834b-429e-9fa8-f7b67d6cde39 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:35.073350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c464ce-834b-429e-9fa8-f7b67d6cde39 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:35.078626 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692fcea-b509-4031-bfc9-bd429b49fd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:35.079221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692fcea-b509-4031-bfc9-bd429b49fd63 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:35.079501 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692fcea-b509-4031-bfc9-bd429b49fd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:35.079776 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692fcea-b509-4031-bfc9-bd429b49fd63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:35.093472 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692fcea-b509-4031-bfc9-bd429b49fd63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:35.093472 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692fcea-b509-4031-bfc9-bd429b49fd63 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:35.093746 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1032/2070] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:35.097988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208169e0-fae6-4892-ad5f-e025c80db804 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:35.108572 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208169e0-fae6-4892-ad5f-e025c80db804 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:35.113377 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796868b6-d5e8-4322-8691-cf179dc665d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:35.114027 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796868b6-d5e8-4322-8691-cf179dc665d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:35.114310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796868b6-d5e8-4322-8691-cf179dc665d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:35.114588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796868b6-d5e8-4322-8691-cf179dc665d4 admin admin] PATH_INFO: `/v3/projects/ca0bc6b0f03945cea752e08c19516031/users/9653b0c1ac9e42f498cf09ae6133bcbb/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:35.131938 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796868b6-d5e8-4322-8691-cf179dc665d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca0bc6b0f03945cea752e08c19516031, user_id=9653b0c1ac9e42f498cf09ae6133bcbb, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:35.135095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796868b6-d5e8-4322-8691-cf179dc665d4 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:35.146150 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692fcea-b509-4031-bfc9-bd429b49fd63 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:35.147996 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b692fcea-b509-4031-bfc9-bd429b49fd63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:35.175182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692fcea-b509-4031-bfc9-bd429b49fd63 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:35.182468 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a0fe4-c5c7-4174-9287-4171cfe7a2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:35.183114 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a0fe4-c5c7-4174-9287-4171cfe7a2ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:35.183393 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a0fe4-c5c7-4174-9287-4171cfe7a2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:35.183667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a0fe4-c5c7-4174-9287-4171cfe7a2ea admin admin] PATH_INFO: `/v3/projects/5765abd4468441e790c614f3982279cc` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:35.184838 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a0fe4-c5c7-4174-9287-4171cfe7a2ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=5765abd4468441e790c614f3982279cc)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:35.187646 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a0fe4-c5c7-4174-9287-4171cfe7a2ea admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:35.236570 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1039/2071] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:35 2022] PUT /identity/v3/projects/ca0bc6b0f03945cea752e08c19516031/users/9653b0c1ac9e42f498cf09ae6133bcbb/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:35.237531 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796868b6-d5e8-4322-8691-cf179dc665d4 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:35.239296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-796868b6-d5e8-4322-8691-cf179dc665d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:35.309804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796868b6-d5e8-4322-8691-cf179dc665d4 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:35.316752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-550a0fe4-c5c7-4174-9287-4171cfe7a2ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5765abd4468441e790c614f3982279cc', 'request_id': 'req-550a0fe4-c5c7-4174-9287-4171cfe7a2ea'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:35.322532 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-550a0fe4-c5c7-4174-9287-4171cfe7a2ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5765abd4468441e790c614f3982279cc', 'request_id': 'req-550a0fe4-c5c7-4174-9287-4171cfe7a2ea'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:35.329196 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-550a0fe4-c5c7-4174-9287-4171cfe7a2ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5765abd4468441e790c614f3982279cc', 'request_id': 'req-550a0fe4-c5c7-4174-9287-4171cfe7a2ea'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:35.369107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce69ce-e92e-42b6-9f07-da8875ea2552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:35.369917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce69ce-e92e-42b6-9f07-da8875ea2552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:35.370437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce69ce-e92e-42b6-9f07-da8875ea2552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:35.370532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce69ce-e92e-42b6-9f07-da8875ea2552 admin admin] PATH_INFO: `/v3/projects/fb1fe081e60a494082806e3d8aa6fed7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:35.371608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce69ce-e92e-42b6-9f07-da8875ea2552 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb1fe081e60a494082806e3d8aa6fed7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:35.373554 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce69ce-e92e-42b6-9f07-da8875ea2552 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:35.409697 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-550a0fe4-c5c7-4174-9287-4171cfe7a2ea admin admin] The token cache is being invalidate because project 5765abd4468441e790c614f3982279cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:35.409990 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-550a0fe4-c5c7-4174-9287-4171cfe7a2ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:35.410733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-550a0fe4-c5c7-4174-9287-4171cfe7a2ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:35.412447 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1033/2072] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:35 2022] DELETE /identity/v3/projects/5765abd4468441e790c614f3982279cc => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:35.414372 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06f217-e3a5-4ac7-bc5f-92a2a9f2482c None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:35.415401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06f217-e3a5-4ac7-bc5f-92a2a9f2482c None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:35.415784 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06f217-e3a5-4ac7-bc5f-92a2a9f2482c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:35.435493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-ce06f217-e3a5-4ac7-bc5f-92a2a9f2482c None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:35.436716 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-ce06f217-e3a5-4ac7-bc5f-92a2a9f2482c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:35.444298 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1034/2073] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:35.445660 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56583e02-a84e-4762-8e92-86461d127ab8 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:35.445919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56583e02-a84e-4762-8e92-86461d127ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:35.449937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56583e02-a84e-4762-8e92-86461d127ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:35.468855 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-56583e02-a84e-4762-8e92-86461d127ab8 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:35.473003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-56583e02-a84e-4762-8e92-86461d127ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:35.478752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1035/2074] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:35.479538 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56583e02-a84e-4762-8e92-86461d127ab8 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:35.480988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-56583e02-a84e-4762-8e92-86461d127ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:35.488983 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-63ce69ce-e92e-42b6-9f07-da8875ea2552 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb1fe081e60a494082806e3d8aa6fed7', 'request_id': 'req-63ce69ce-e92e-42b6-9f07-da8875ea2552'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:35.495017 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-63ce69ce-e92e-42b6-9f07-da8875ea2552 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb1fe081e60a494082806e3d8aa6fed7', 'request_id': 'req-63ce69ce-e92e-42b6-9f07-da8875ea2552'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:35.501433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-63ce69ce-e92e-42b6-9f07-da8875ea2552 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb1fe081e60a494082806e3d8aa6fed7', 'request_id': 'req-63ce69ce-e92e-42b6-9f07-da8875ea2552'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:35.537777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56583e02-a84e-4762-8e92-86461d127ab8 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:35.550860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:35.584471 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-63ce69ce-e92e-42b6-9f07-da8875ea2552 admin admin] The token cache is being invalidate because project fb1fe081e60a494082806e3d8aa6fed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:35.584870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-63ce69ce-e92e-42b6-9f07-da8875ea2552 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:35.585473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-63ce69ce-e92e-42b6-9f07-da8875ea2552 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:35.587486 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1040/2075] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:35 2022] DELETE /identity/v3/projects/fb1fe081e60a494082806e3d8aa6fed7 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:35.593992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42936666-4ac3-4f10-aec5-cc9a185b4702 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:35.593992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42936666-4ac3-4f10-aec5-cc9a185b4702 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:35.593992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42936666-4ac3-4f10-aec5-cc9a185b4702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:35.594230 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0777d4-f917-4a5e-bdb6-8798feb76646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:35.594230 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0777d4-f917-4a5e-bdb6-8798feb76646 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:35.594230 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0777d4-f917-4a5e-bdb6-8798feb76646 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:35.594230 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0777d4-f917-4a5e-bdb6-8798feb76646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:35.595005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4f0777d4-f917-4a5e-bdb6-8798feb76646 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:35.611080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-42936666-4ac3-4f10-aec5-cc9a185b4702 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:35.616989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-42936666-4ac3-4f10-aec5-cc9a185b4702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:35.665724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0777d4-f917-4a5e-bdb6-8798feb76646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:35.667434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0777d4-f917-4a5e-bdb6-8798feb76646 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:35.678699 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1036/2076] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:35.679269 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1041/2077] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:35.679549 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0777d4-f917-4a5e-bdb6-8798feb76646 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:35.681772 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4f0777d4-f917-4a5e-bdb6-8798feb76646 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:35.684277 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42936666-4ac3-4f10-aec5-cc9a185b4702 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:35.695200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42936666-4ac3-4f10-aec5-cc9a185b4702 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:35.698243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0777d4-f917-4a5e-bdb6-8798feb76646 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:35.700663 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380769b2-b4b3-45e1-bfaa-9a6711ba2376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:35.701307 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380769b2-b4b3-45e1-bfaa-9a6711ba2376 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:35.701621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380769b2-b4b3-45e1-bfaa-9a6711ba2376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:35.701901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380769b2-b4b3-45e1-bfaa-9a6711ba2376 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:35.702675 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380769b2-b4b3-45e1-bfaa-9a6711ba2376 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:35.702973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e53253-bf59-4641-a189-40c5e73ea3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:35.703111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380769b2-b4b3-45e1-bfaa-9a6711ba2376 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:35.703610 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e53253-bf59-4641-a189-40c5e73ea3e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:35.703893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e53253-bf59-4641-a189-40c5e73ea3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:35.704176 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e53253-bf59-4641-a189-40c5e73ea3e0 admin admin] PATH_INFO: `/v3/users/9d1a3b18bf614d639c2440a7a490ce0b` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:35.704592 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380769b2-b4b3-45e1-bfaa-9a6711ba2376 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:35.705338 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e53253-bf59-4641-a189-40c5e73ea3e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d1a3b18bf614d639c2440a7a490ce0b)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:35.707304 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e53253-bf59-4641-a189-40c5e73ea3e0 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:35.717592 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1042/2078] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:37:35.721964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380769b2-b4b3-45e1-bfaa-9a6711ba2376 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:35.748912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380769b2-b4b3-45e1-bfaa-9a6711ba2376 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:35.754258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ec353-a03e-4677-8ecf-09ca99a33951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:35.755048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ec353-a03e-4677-8ecf-09ca99a33951 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:35.755575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ec353-a03e-4677-8ecf-09ca99a33951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:35.755963 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ec353-a03e-4677-8ecf-09ca99a33951 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:35.756709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ec353-a03e-4677-8ecf-09ca99a33951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:35.759995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ec353-a03e-4677-8ecf-09ca99a33951 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:35.850670 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1043/2079] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:35 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:37:35.851837 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ec353-a03e-4677-8ecf-09ca99a33951 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:35.883539 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-e8e53253-bf59-4641-a189-40c5e73ea3e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d1a3b18bf614d639c2440a7a490ce0b'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:35.898495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:35.919110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ec353-a03e-4677-8ecf-09ca99a33951 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:35.923943 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610abdc4-7e09-4135-a158-c6e3b1691d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:35.924548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610abdc4-7e09-4135-a158-c6e3b1691d8c admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:35.924826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610abdc4-7e09-4135-a158-c6e3b1691d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:35.925136 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610abdc4-7e09-4135-a158-c6e3b1691d8c admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:35.925771 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610abdc4-7e09-4135-a158-c6e3b1691d8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:35.927674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610abdc4-7e09-4135-a158-c6e3b1691d8c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:35.983584 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-e8e53253-bf59-4641-a189-40c5e73ea3e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d1a3b18bf614d639c2440a7a490ce0b'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:35.984218 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-e8e53253-bf59-4641-a189-40c5e73ea3e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d1a3b18bf614d639c2440a7a490ce0b'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:35.984874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-e8e53253-bf59-4641-a189-40c5e73ea3e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d1a3b18bf614d639c2440a7a490ce0b'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:35.992295 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-e8e53253-bf59-4641-a189-40c5e73ea3e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d1a3b18bf614d639c2440a7a490ce0b'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:36.011684 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1037/2080] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:35 2022] DELETE /identity/v3/users/9d1a3b18bf614d639c2440a7a490ce0b => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:36.037222 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1044/2081] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:35 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:36.038385 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e53253-bf59-4641-a189-40c5e73ea3e0 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:36.040597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e8e53253-bf59-4641-a189-40c5e73ea3e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:36.119985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e53253-bf59-4641-a189-40c5e73ea3e0 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:36.125439 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcebe7-0970-495b-83d0-6c9128fd14c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:36.126200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcebe7-0970-495b-83d0-6c9128fd14c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:36.126540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcebe7-0970-495b-83d0-6c9128fd14c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:36.126920 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcebe7-0970-495b-83d0-6c9128fd14c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:36.127665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bcebe7-0970-495b-83d0-6c9128fd14c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:36.129450 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bcebe7-0970-495b-83d0-6c9128fd14c5 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:36.140436 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1038/2082] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:36.145704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610abdc4-7e09-4135-a158-c6e3b1691d8c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:36.155243 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610abdc4-7e09-4135-a158-c6e3b1691d8c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:36.160248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9c934-827b-4806-aad9-8ff0f3d09597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:36.160852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9c934-827b-4806-aad9-8ff0f3d09597 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:36.161168 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9c934-827b-4806-aad9-8ff0f3d09597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:36.161447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9c934-827b-4806-aad9-8ff0f3d09597 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:36.162094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9c934-827b-4806-aad9-8ff0f3d09597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:36.163799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9c934-827b-4806-aad9-8ff0f3d09597 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:36.174515 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1045/2083] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:36.180161 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcebe7-0970-495b-83d0-6c9128fd14c5 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:36.191049 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcebe7-0970-495b-83d0-6c9128fd14c5 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:36.196070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60738a71-194f-4851-932b-99f4d13eb98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:36.196784 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60738a71-194f-4851-932b-99f4d13eb98a admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:36.197090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60738a71-194f-4851-932b-99f4d13eb98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:36.197369 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60738a71-194f-4851-932b-99f4d13eb98a admin admin] PATH_INFO: `/v3/projects/01fa72bebe0f4f8283a98e3553e997e3/users/e3c892f5837340258eac011d84bcac22/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:36.215091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60738a71-194f-4851-932b-99f4d13eb98a admin admin] RBAC: Authorizing `identity:create_grant(project_id=01fa72bebe0f4f8283a98e3553e997e3, user_id=e3c892f5837340258eac011d84bcac22, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:36.217934 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60738a71-194f-4851-932b-99f4d13eb98a admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:36.281682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:36.289307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1039/2084] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:36 2022] PUT /identity/v3/projects/01fa72bebe0f4f8283a98e3553e997e3/users/e3c892f5837340258eac011d84bcac22/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:36.300454 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50c591-7ee3-4de9-b9e2-497cf0e69e56 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:36.300738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50c591-7ee3-4de9-b9e2-497cf0e69e56 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:36.301013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50c591-7ee3-4de9-b9e2-497cf0e69e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:36.330194 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-9f50c591-7ee3-4de9-b9e2-497cf0e69e56 None None] MFA Rules not processed for user `e3c892f5837340258eac011d84bcac22`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:36.331530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9f50c591-7ee3-4de9-b9e2-497cf0e69e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:36.412651 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1046/2085] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:36.417770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60738a71-194f-4851-932b-99f4d13eb98a admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:36.491190 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60738a71-194f-4851-932b-99f4d13eb98a admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:36.499856 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7528a44-568d-49c0-95f5-fbf5320d6553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:36.501331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7528a44-568d-49c0-95f5-fbf5320d6553 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:36.501976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7528a44-568d-49c0-95f5-fbf5320d6553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:36.502944 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7528a44-568d-49c0-95f5-fbf5320d6553 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:36.503873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7528a44-568d-49c0-95f5-fbf5320d6553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:36.506445 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7528a44-568d-49c0-95f5-fbf5320d6553 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:36.619358 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:36.620594 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50c591-7ee3-4de9-b9e2-497cf0e69e56 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:36.622707 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9f50c591-7ee3-4de9-b9e2-497cf0e69e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:36.642522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1040/2086] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:36 2022] POST /identity/v3/projects => generated 377 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:36.643351 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7528a44-568d-49c0-95f5-fbf5320d6553 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:36.662189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50c591-7ee3-4de9-b9e2-497cf0e69e56 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:36.727450 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8dbe91-575d-4d1f-81f5-5eaf966b085d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:36.728845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8dbe91-575d-4d1f-81f5-5eaf966b085d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:36.729162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8dbe91-575d-4d1f-81f5-5eaf966b085d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:36.729679 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8dbe91-575d-4d1f-81f5-5eaf966b085d admin admin] PATH_INFO: `/v3/projects/1c7253d01b034f7390e374132c671216` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:36.731645 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8dbe91-575d-4d1f-81f5-5eaf966b085d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c7253d01b034f7390e374132c671216)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:36.734839 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8dbe91-575d-4d1f-81f5-5eaf966b085d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:36.742158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7528a44-568d-49c0-95f5-fbf5320d6553 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:36.748406 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df43c7-15c5-4bf0-88b0-0a6e87af0ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:36.749025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df43c7-15c5-4bf0-88b0-0a6e87af0ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:36.749340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df43c7-15c5-4bf0-88b0-0a6e87af0ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:36.749614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df43c7-15c5-4bf0-88b0-0a6e87af0ba6 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:36.750252 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df43c7-15c5-4bf0-88b0-0a6e87af0ba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:36.752215 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df43c7-15c5-4bf0-88b0-0a6e87af0ba6 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:36.868238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1041/2087] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:36 2022] POST /identity/v3/users => generated 289 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:36.869233 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df43c7-15c5-4bf0-88b0-0a6e87af0ba6 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:36.870221 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7c8dbe91-575d-4d1f-81f5-5eaf966b085d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c7253d01b034f7390e374132c671216', 'request_id': 'req-7c8dbe91-575d-4d1f-81f5-5eaf966b085d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:36.877432 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7c8dbe91-575d-4d1f-81f5-5eaf966b085d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c7253d01b034f7390e374132c671216', 'request_id': 'req-7c8dbe91-575d-4d1f-81f5-5eaf966b085d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:36.881731 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df43c7-15c5-4bf0-88b0-0a6e87af0ba6 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:36.892260 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7c8dbe91-575d-4d1f-81f5-5eaf966b085d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c7253d01b034f7390e374132c671216', 'request_id': 'req-7c8dbe91-575d-4d1f-81f5-5eaf966b085d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:36.919152 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:36.944541 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdb1c0-8d31-4837-8113-e55f3650edc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:36.945470 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdb1c0-8d31-4837-8113-e55f3650edc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:36.946522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdb1c0-8d31-4837-8113-e55f3650edc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:36.947426 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdb1c0-8d31-4837-8113-e55f3650edc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:36.948724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fdb1c0-8d31-4837-8113-e55f3650edc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:36.949887 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7c8dbe91-575d-4d1f-81f5-5eaf966b085d admin admin] The token cache is being invalidate because project 1c7253d01b034f7390e374132c671216 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:36.950340 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7c8dbe91-575d-4d1f-81f5-5eaf966b085d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:36.951238 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7c8dbe91-575d-4d1f-81f5-5eaf966b085d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:36.952287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fdb1c0-8d31-4837-8113-e55f3650edc5 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:36.956856 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1047/2088] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:36 2022] DELETE /identity/v3/projects/1c7253d01b034f7390e374132c671216 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:36.958795 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb648a-d83c-4910-8476-23efea93feda None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:36.959153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb648a-d83c-4910-8476-23efea93feda None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:36.959436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb648a-d83c-4910-8476-23efea93feda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:36.972059 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1042/2089] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:36.973640 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdb1c0-8d31-4837-8113-e55f3650edc5 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:36.975495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-37fdb1c0-8d31-4837-8113-e55f3650edc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:36.987642 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-dceb648a-d83c-4910-8476-23efea93feda None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:36.988864 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-dceb648a-d83c-4910-8476-23efea93feda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:36.990525 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdb1c0-8d31-4837-8113-e55f3650edc5 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:36.996074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ce4d0-1bf5-4435-84ad-71dec0735706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:36.996760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ce4d0-1bf5-4435-84ad-71dec0735706 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:36.997193 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ce4d0-1bf5-4435-84ad-71dec0735706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:36.997570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ce4d0-1bf5-4435-84ad-71dec0735706 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:36.999708 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ce4d0-1bf5-4435-84ad-71dec0735706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:37.002833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ce4d0-1bf5-4435-84ad-71dec0735706 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:37.008994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1048/2090] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:37.016836 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111257f-7bb3-4283-9f90-4a151ae7126c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:37.017139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111257f-7bb3-4283-9f90-4a151ae7126c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:37.017431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111257f-7bb3-4283-9f90-4a151ae7126c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:37.021880 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1043/2091] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:37.026005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ce4d0-1bf5-4435-84ad-71dec0735706 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:37.036446 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ce4d0-1bf5-4435-84ad-71dec0735706 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:37.037718 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1111257f-7bb3-4283-9f90-4a151ae7126c None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:37.038708 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1111257f-7bb3-4283-9f90-4a151ae7126c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:37.041464 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13d849-2889-4a64-bc67-bfd23a25c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:37.042210 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13d849-2889-4a64-bc67-bfd23a25c442 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:37.042557 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13d849-2889-4a64-bc67-bfd23a25c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:37.042939 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13d849-2889-4a64-bc67-bfd23a25c442 admin admin] PATH_INFO: `/v3/projects/75f73f1c70b5444b88f81bf5f3868f13/users/c316f393e0fb45b785cd65b069942808/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:37.047159 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1049/2092] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:37.051845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111257f-7bb3-4283-9f90-4a151ae7126c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:37.064335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111257f-7bb3-4283-9f90-4a151ae7126c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:37.080458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19811eb-cd03-4da5-b367-3c5dffd50ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:37.081105 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19811eb-cd03-4da5-b367-3c5dffd50ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:37.081388 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19811eb-cd03-4da5-b367-3c5dffd50ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:37.081640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19811eb-cd03-4da5-b367-3c5dffd50ab8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:37.082299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19811eb-cd03-4da5-b367-3c5dffd50ab8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:37.082599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19811eb-cd03-4da5-b367-3c5dffd50ab8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:37.084336 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19811eb-cd03-4da5-b367-3c5dffd50ab8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:37.086507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13d849-2889-4a64-bc67-bfd23a25c442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f73f1c70b5444b88f81bf5f3868f13, user_id=c316f393e0fb45b785cd65b069942808, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:37.089675 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13d849-2889-4a64-bc67-bfd23a25c442 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:37.096867 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1050/2093] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:37:37.101530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19811eb-cd03-4da5-b367-3c5dffd50ab8 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:37.112180 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19811eb-cd03-4da5-b367-3c5dffd50ab8 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:37.118651 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584d2b7-39f9-467c-a1e4-08f90bc160b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:37.119269 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584d2b7-39f9-467c-a1e4-08f90bc160b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:37.119771 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584d2b7-39f9-467c-a1e4-08f90bc160b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:37.119881 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584d2b7-39f9-467c-a1e4-08f90bc160b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:37.120501 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584d2b7-39f9-467c-a1e4-08f90bc160b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:37.122472 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584d2b7-39f9-467c-a1e4-08f90bc160b1 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:37.220363 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1044/2094] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:37 2022] PUT /identity/v3/projects/75f73f1c70b5444b88f81bf5f3868f13/users/c316f393e0fb45b785cd65b069942808/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:37.230988 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff521e61-a2e6-4223-938e-2a2666229a5e None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:37.231329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff521e61-a2e6-4223-938e-2a2666229a5e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:37.231618 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff521e61-a2e6-4223-938e-2a2666229a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:37.251457 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1051/2095] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:37 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:37:37.253093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e2a45-8105-4ee2-9e3b-1b9aa051684a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:37.253469 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e2a45-8105-4ee2-9e3b-1b9aa051684a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:37.253943 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e2a45-8105-4ee2-9e3b-1b9aa051684a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:37.264735 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-ff521e61-a2e6-4223-938e-2a2666229a5e None None] MFA Rules not processed for user `9653b0c1ac9e42f498cf09ae6133bcbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:37.266350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-ff521e61-a2e6-4223-938e-2a2666229a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:37.284236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-3b9e2a45-8105-4ee2-9e3b-1b9aa051684a None None] MFA Rules not processed for user `c316f393e0fb45b785cd65b069942808`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:37.285499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3b9e2a45-8105-4ee2-9e3b-1b9aa051684a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:37.361096 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1045/2096] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:37.362816 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff521e61-a2e6-4223-938e-2a2666229a5e None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:37.378839 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1052/2097] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:37.385162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e2a45-8105-4ee2-9e3b-1b9aa051684a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:37.387070 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3b9e2a45-8105-4ee2-9e3b-1b9aa051684a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:37.389836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:37.436994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff521e61-a2e6-4223-938e-2a2666229a5e None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:37.441951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a27a78-d7fb-42fb-92fc-1493c5a101af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:37.442562 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a27a78-d7fb-42fb-92fc-1493c5a101af admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:37.442847 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a27a78-d7fb-42fb-92fc-1493c5a101af admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:37.443120 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a27a78-d7fb-42fb-92fc-1493c5a101af admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:37.443781 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a27a78-d7fb-42fb-92fc-1493c5a101af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:37.446421 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a27a78-d7fb-42fb-92fc-1493c5a101af admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:37.460698 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e2a45-8105-4ee2-9e3b-1b9aa051684a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:37.466046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f0fe4-b5a0-4184-b892-47c7c7551fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:37.466657 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f0fe4-b5a0-4184-b892-47c7c7551fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:37.466937 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f0fe4-b5a0-4184-b892-47c7c7551fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:37.467213 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f0fe4-b5a0-4184-b892-47c7c7551fb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:37.467852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f0fe4-b5a0-4184-b892-47c7c7551fb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:37.469797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f0fe4-b5a0-4184-b892-47c7c7551fb9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:37.515950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:37.547698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1046/2098] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:37 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:37.548693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a27a78-d7fb-42fb-92fc-1493c5a101af admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:37.549781 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1053/2099] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:37 2022] POST /identity/v3/projects => generated 373 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:37.552336 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f0fe4-b5a0-4184-b892-47c7c7551fb9 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:37.559207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a27a78-d7fb-42fb-92fc-1493c5a101af admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:37.613604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f0fe4-b5a0-4184-b892-47c7c7551fb9 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:37.616119 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc0163-cd73-41cf-ad53-3e83d9a2f5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:37.616751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc0163-cd73-41cf-ad53-3e83d9a2f5b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:37.617276 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc0163-cd73-41cf-ad53-3e83d9a2f5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:37.617623 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc0163-cd73-41cf-ad53-3e83d9a2f5b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:37.618731 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc0163-cd73-41cf-ad53-3e83d9a2f5b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:37.619021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad5abe-8815-4702-b72b-1c9452a5d020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:37.619747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad5abe-8815-4702-b72b-1c9452a5d020 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:37.620033 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad5abe-8815-4702-b72b-1c9452a5d020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:37.620310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad5abe-8815-4702-b72b-1c9452a5d020 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:37.620870 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc0163-cd73-41cf-ad53-3e83d9a2f5b4 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:37.620975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad5abe-8815-4702-b72b-1c9452a5d020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:37.622564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad5abe-8815-4702-b72b-1c9452a5d020 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:37.632701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1054/2100] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:37.634413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad5abe-8815-4702-b72b-1c9452a5d020 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:37.644148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad5abe-8815-4702-b72b-1c9452a5d020 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:37.649597 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c8482-2e5a-4345-90d1-d47cf5955421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:37.650198 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c8482-2e5a-4345-90d1-d47cf5955421 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:37.650478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c8482-2e5a-4345-90d1-d47cf5955421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:37.650748 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c8482-2e5a-4345-90d1-d47cf5955421 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:37.651378 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c8482-2e5a-4345-90d1-d47cf5955421 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:37.653328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c8482-2e5a-4345-90d1-d47cf5955421 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:37.694720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1047/2101] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:37 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:37.694720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc0163-cd73-41cf-ad53-3e83d9a2f5b4 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:37.754085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc0163-cd73-41cf-ad53-3e83d9a2f5b4 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:37.761098 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db3525-db43-4747-843c-b1a8339206d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:37.761710 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db3525-db43-4747-843c-b1a8339206d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:37.761989 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db3525-db43-4747-843c-b1a8339206d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:37.762258 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db3525-db43-4747-843c-b1a8339206d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:37.762926 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db3525-db43-4747-843c-b1a8339206d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:37.764589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db3525-db43-4747-843c-b1a8339206d7 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:37.770461 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1055/2102] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:37 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:37.772747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:37.778506 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c8482-2e5a-4345-90d1-d47cf5955421 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:37.778657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1048/2103] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:37.778657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db3525-db43-4747-843c-b1a8339206d7 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:37.785593 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c8482-2e5a-4345-90d1-d47cf5955421 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:37.786570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db3525-db43-4747-843c-b1a8339206d7 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:37.790803 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6707190-8c6c-4168-9edc-0ec28efe93ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:37.791465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6707190-8c6c-4168-9edc-0ec28efe93ce admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:37.791595 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca1048-2079-497b-b751-1f7156f927b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:37.791794 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6707190-8c6c-4168-9edc-0ec28efe93ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:37.792034 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6707190-8c6c-4168-9edc-0ec28efe93ce admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:37.792254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca1048-2079-497b-b751-1f7156f927b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:37.792594 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca1048-2079-497b-b751-1f7156f927b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:37.792697 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6707190-8c6c-4168-9edc-0ec28efe93ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:37.792891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca1048-2079-497b-b751-1f7156f927b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:37.793549 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca1048-2079-497b-b751-1f7156f927b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:37.794753 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6707190-8c6c-4168-9edc-0ec28efe93ce admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:37.795194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca1048-2079-497b-b751-1f7156f927b1 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:37.806921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1049/2104] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:37.806921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca1048-2079-497b-b751-1f7156f927b1 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:37.817071 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca1048-2079-497b-b751-1f7156f927b1 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:37.822508 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca55b9a-e801-4d6d-ba56-fede802806bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:37.823162 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca55b9a-e801-4d6d-ba56-fede802806bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:37.823443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca55b9a-e801-4d6d-ba56-fede802806bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:37.823721 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca55b9a-e801-4d6d-ba56-fede802806bf admin admin] PATH_INFO: `/v3/projects/a2317296ebf349869d03736c7434396a/users/1728e7e5f236414ca5ec39e99004a8be/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:37.839040 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca55b9a-e801-4d6d-ba56-fede802806bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2317296ebf349869d03736c7434396a, user_id=1728e7e5f236414ca5ec39e99004a8be, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:37.841783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca55b9a-e801-4d6d-ba56-fede802806bf admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:37.861733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:37.893654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1056/2105] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:37 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:37.894547 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6707190-8c6c-4168-9edc-0ec28efe93ce admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:37.903982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6707190-8c6c-4168-9edc-0ec28efe93ce admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:37.908516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8669e3-2c14-403a-b4d7-e2df5383496c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:37.909434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8669e3-2c14-403a-b4d7-e2df5383496c admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:37.914075 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8669e3-2c14-403a-b4d7-e2df5383496c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:37.914075 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8669e3-2c14-403a-b4d7-e2df5383496c admin admin] PATH_INFO: `/v3/projects/454cc1021dbe4735870f4146232b5fb6/users/37554a0aebaf4dd486aeeb4ce830d48a/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:37.915758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1050/2106] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:37 2022] PUT /identity/v3/projects/a2317296ebf349869d03736c7434396a/users/1728e7e5f236414ca5ec39e99004a8be/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:37.916440 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca55b9a-e801-4d6d-ba56-fede802806bf admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:37.925460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8669e3-2c14-403a-b4d7-e2df5383496c admin admin] RBAC: Authorizing `identity:create_grant(project_id=454cc1021dbe4735870f4146232b5fb6, user_id=37554a0aebaf4dd486aeeb4ce830d48a, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:37.928130 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8669e3-2c14-403a-b4d7-e2df5383496c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:37.981214 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca55b9a-e801-4d6d-ba56-fede802806bf admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:37.985953 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264e2dc-7c93-4e8b-a7cc-9799531923ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:37.986556 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264e2dc-7c93-4e8b-a7cc-9799531923ca admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:37.986841 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264e2dc-7c93-4e8b-a7cc-9799531923ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:37.987117 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264e2dc-7c93-4e8b-a7cc-9799531923ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:37.987751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264e2dc-7c93-4e8b-a7cc-9799531923ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:37.989514 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264e2dc-7c93-4e8b-a7cc-9799531923ca admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:38.003480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1051/2107] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:38.003480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab3e04-6d5f-412f-bdd8-616fbec8a555 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:38.003480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab3e04-6d5f-412f-bdd8-616fbec8a555 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:38.003480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab3e04-6d5f-412f-bdd8-616fbec8a555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:38.025144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1057/2108] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:37 2022] PUT /identity/v3/projects/454cc1021dbe4735870f4146232b5fb6/users/37554a0aebaf4dd486aeeb4ce830d48a/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:38.025980 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8669e3-2c14-403a-b4d7-e2df5383496c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:38.030277 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-12ab3e04-6d5f-412f-bdd8-616fbec8a555 None None] MFA Rules not processed for user `1728e7e5f236414ca5ec39e99004a8be`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:38.031827 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-12ab3e04-6d5f-412f-bdd8-616fbec8a555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:38.042755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:38.091769 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8669e3-2c14-403a-b4d7-e2df5383496c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:38.096770 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeafc89c-9202-4b96-a936-19cc8b121e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:38.097474 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeafc89c-9202-4b96-a936-19cc8b121e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:38.097758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeafc89c-9202-4b96-a936-19cc8b121e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:38.098020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeafc89c-9202-4b96-a936-19cc8b121e5f admin admin] PATH_INFO: `/v3/projects/e1a80d6abd504412a8aa993878390b8a/users/59b2fe7773744929aac82cf681e45345/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:38.108073 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1052/2109] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:38.108995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab3e04-6d5f-412f-bdd8-616fbec8a555 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:38.115052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeafc89c-9202-4b96-a936-19cc8b121e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a80d6abd504412a8aa993878390b8a, user_id=59b2fe7773744929aac82cf681e45345, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:38.124142 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeafc89c-9202-4b96-a936-19cc8b121e5f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:38.124299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab3e04-6d5f-412f-bdd8-616fbec8a555 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:38.125702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bd407-b36d-40d6-a609-8c39b524f57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:38.126307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bd407-b36d-40d6-a609-8c39b524f57f admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:38.126588 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bd407-b36d-40d6-a609-8c39b524f57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:38.126862 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bd407-b36d-40d6-a609-8c39b524f57f admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:38.127950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18bd407-b36d-40d6-a609-8c39b524f57f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:38.129593 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18bd407-b36d-40d6-a609-8c39b524f57f admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:38.140259 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1053/2110] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:38.143554 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bd407-b36d-40d6-a609-8c39b524f57f admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:38.151523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bd407-b36d-40d6-a609-8c39b524f57f admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:38.156247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e10746-f24d-4280-81e5-0b01727be6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:38.156809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e10746-f24d-4280-81e5-0b01727be6b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:38.157117 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e10746-f24d-4280-81e5-0b01727be6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:38.157400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e10746-f24d-4280-81e5-0b01727be6b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:38.158007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e10746-f24d-4280-81e5-0b01727be6b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:38.160030 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e10746-f24d-4280-81e5-0b01727be6b4 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:38.203428 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1058/2111] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:38 2022] PUT /identity/v3/projects/e1a80d6abd504412a8aa993878390b8a/users/59b2fe7773744929aac82cf681e45345/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:38.205848 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeafc89c-9202-4b96-a936-19cc8b121e5f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:38.269506 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeafc89c-9202-4b96-a936-19cc8b121e5f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:38.274824 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ce5bf-1c78-4d36-84c2-d23e724c421e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:38.275580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ce5bf-1c78-4d36-84c2-d23e724c421e admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:38.275856 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ce5bf-1c78-4d36-84c2-d23e724c421e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:38.276126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ce5bf-1c78-4d36-84c2-d23e724c421e admin admin] PATH_INFO: `/v3/projects/454cc1021dbe4735870f4146232b5fb6/users/37554a0aebaf4dd486aeeb4ce830d48a/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:38.279705 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1054/2112] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:38 2022] POST /identity/v3/projects => generated 381 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:38.279705 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e10746-f24d-4280-81e5-0b01727be6b4 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:38.281232 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ce5bf-1c78-4d36-84c2-d23e724c421e admin admin] RBAC: Authorizing `identity:create_grant(project_id=454cc1021dbe4735870f4146232b5fb6, user_id=37554a0aebaf4dd486aeeb4ce830d48a, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:38.284018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ce5bf-1c78-4d36-84c2-d23e724c421e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:38.298361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1059/2113] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:38 2022] PUT /identity/v3/projects/454cc1021dbe4735870f4146232b5fb6/users/37554a0aebaf4dd486aeeb4ce830d48a/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:38.299068 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ce5bf-1c78-4d36-84c2-d23e724c421e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:38.299845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:38.356449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e10746-f24d-4280-81e5-0b01727be6b4 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:38.361657 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:38.371547 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ce5bf-1c78-4d36-84c2-d23e724c421e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:38.376465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8eb87c-0b18-49f1-807a-cdb904f06c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:38.377095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8eb87c-0b18-49f1-807a-cdb904f06c30 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:38.377372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8eb87c-0b18-49f1-807a-cdb904f06c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:38.377645 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8eb87c-0b18-49f1-807a-cdb904f06c30 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:38.378324 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8eb87c-0b18-49f1-807a-cdb904f06c30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:38.380206 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8eb87c-0b18-49f1-807a-cdb904f06c30 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:38.420899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dd3ba-7d3c-4edf-9f85-8052b7078b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:38.421558 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dd3ba-7d3c-4edf-9f85-8052b7078b93 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:38.421841 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dd3ba-7d3c-4edf-9f85-8052b7078b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:38.422517 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dd3ba-7d3c-4edf-9f85-8052b7078b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:38.423242 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dd3ba-7d3c-4edf-9f85-8052b7078b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:38.425174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dd3ba-7d3c-4edf-9f85-8052b7078b93 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:38.441520 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1055/2114] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:38.443506 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dd3ba-7d3c-4edf-9f85-8052b7078b93 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:38.456009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dd3ba-7d3c-4edf-9f85-8052b7078b93 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:38.461110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97396116-e44d-4f17-8874-1cf2aad18ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:38.462709 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97396116-e44d-4f17-8874-1cf2aad18ae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:38.462709 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97396116-e44d-4f17-8874-1cf2aad18ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:38.462992 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97396116-e44d-4f17-8874-1cf2aad18ae8 admin admin] PATH_INFO: `/v3/projects/e1a80d6abd504412a8aa993878390b8a/users/59b2fe7773744929aac82cf681e45345/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:38.465306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97396116-e44d-4f17-8874-1cf2aad18ae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a80d6abd504412a8aa993878390b8a, user_id=59b2fe7773744929aac82cf681e45345, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:38.469899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97396116-e44d-4f17-8874-1cf2aad18ae8 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:38.483666 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1056/2115] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:38 2022] PUT /identity/v3/projects/e1a80d6abd504412a8aa993878390b8a/users/59b2fe7773744929aac82cf681e45345/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:38.494879 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f705f-7630-4d51-9b86-dc5f38fda3ae None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:38.495166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f705f-7630-4d51-9b86-dc5f38fda3ae None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:38.495458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f705f-7630-4d51-9b86-dc5f38fda3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:38.505097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1060/2116] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:38 2022] POST /identity/v3/users => generated 297 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:38.507142 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8eb87c-0b18-49f1-807a-cdb904f06c30 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:38.528435 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8eb87c-0b18-49f1-807a-cdb904f06c30 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:38.529664 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-c79f705f-7630-4d51-9b86-dc5f38fda3ae None None] MFA Rules not processed for user `59b2fe7773744929aac82cf681e45345`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:38.531191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c79f705f-7630-4d51-9b86-dc5f38fda3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:38.595990 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979a05a-52e5-4431-969f-16a471150946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:38.597049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979a05a-52e5-4431-969f-16a471150946 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:38.597359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979a05a-52e5-4431-969f-16a471150946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:38.597712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979a05a-52e5-4431-969f-16a471150946 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:38.598367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979a05a-52e5-4431-969f-16a471150946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:38.600059 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979a05a-52e5-4431-969f-16a471150946 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:38.608444 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1057/2117] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:38.612665 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1061/2118] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:38.617342 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f705f-7630-4d51-9b86-dc5f38fda3ae None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:38.626627 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979a05a-52e5-4431-969f-16a471150946 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:38.628585 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1979a05a-52e5-4431-969f-16a471150946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:38.630263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f705f-7630-4d51-9b86-dc5f38fda3ae None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:38.636059 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbf8ba-27ea-4f08-a8c9-6d9a15279fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:38.637273 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbf8ba-27ea-4f08-a8c9-6d9a15279fba admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:38.637273 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbf8ba-27ea-4f08-a8c9-6d9a15279fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:38.637471 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbf8ba-27ea-4f08-a8c9-6d9a15279fba admin admin] PATH_INFO: `/v3/projects/4fa7755b3fa14b99bcdcd4ac17c476ef/users/a4f712a8587f4ac0a3f5a31ccac44683/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:38.656490 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dbf8ba-27ea-4f08-a8c9-6d9a15279fba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa7755b3fa14b99bcdcd4ac17c476ef, user_id=a4f712a8587f4ac0a3f5a31ccac44683, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:38.659353 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dbf8ba-27ea-4f08-a8c9-6d9a15279fba admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:38.667180 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:38.701753 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979a05a-52e5-4431-969f-16a471150946 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:38.705436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4394c33c-28d1-45a8-b1ef-94dc1968a182 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:38.706036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4394c33c-28d1-45a8-b1ef-94dc1968a182 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:38.706330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4394c33c-28d1-45a8-b1ef-94dc1968a182 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:38.706582 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4394c33c-28d1-45a8-b1ef-94dc1968a182 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:38.708365 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4394c33c-28d1-45a8-b1ef-94dc1968a182 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:38.767290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:38.783570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1058/2119] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:38 2022] PUT /identity/v3/projects/4fa7755b3fa14b99bcdcd4ac17c476ef/users/a4f712a8587f4ac0a3f5a31ccac44683/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:38.787472 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbf8ba-27ea-4f08-a8c9-6d9a15279fba admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:38.790293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4394c33c-28d1-45a8-b1ef-94dc1968a182 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:38.795143 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4394c33c-28d1-45a8-b1ef-94dc1968a182 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:38.802485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1062/2120] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:37:38.874722 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbf8ba-27ea-4f08-a8c9-6d9a15279fba admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:38.879572 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9f153-7004-48b2-8719-acd205756e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:38.880208 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9f153-7004-48b2-8719-acd205756e7a admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:38.880490 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9f153-7004-48b2-8719-acd205756e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:38.880803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9f153-7004-48b2-8719-acd205756e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:38.881454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9f153-7004-48b2-8719-acd205756e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:38.884326 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9f153-7004-48b2-8719-acd205756e7a admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:38.895482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1059/2121] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:38.900190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4394c33c-28d1-45a8-b1ef-94dc1968a182 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:38.913924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4394c33c-28d1-45a8-b1ef-94dc1968a182 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:38.920339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d97654-73c4-4271-92a6-1da42d18a3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:38.921007 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d97654-73c4-4271-92a6-1da42d18a3ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:38.921505 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d97654-73c4-4271-92a6-1da42d18a3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:38.921790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d97654-73c4-4271-92a6-1da42d18a3ce admin admin] PATH_INFO: `/v3/projects/4fa7755b3fa14b99bcdcd4ac17c476ef/users/a4f712a8587f4ac0a3f5a31ccac44683/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:38.924537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d97654-73c4-4271-92a6-1da42d18a3ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa7755b3fa14b99bcdcd4ac17c476ef, user_id=a4f712a8587f4ac0a3f5a31ccac44683, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:38.926706 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d97654-73c4-4271-92a6-1da42d18a3ce admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:38.941093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1063/2122] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:38 2022] PUT /identity/v3/projects/4fa7755b3fa14b99bcdcd4ac17c476ef/users/a4f712a8587f4ac0a3f5a31ccac44683/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:38.956741 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954107d-2021-4df3-a9bf-05716035c9d3 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:38.957027 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954107d-2021-4df3-a9bf-05716035c9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:38.957348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954107d-2021-4df3-a9bf-05716035c9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:38.987916 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-7954107d-2021-4df3-a9bf-05716035c9d3 None None] MFA Rules not processed for user `a4f712a8587f4ac0a3f5a31ccac44683`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:38.989293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-7954107d-2021-4df3-a9bf-05716035c9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:39.065508 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1060/2123] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:38 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:39.078282 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d97654-73c4-4271-92a6-1da42d18a3ce admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:39.081011 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-47d97654-73c4-4271-92a6-1da42d18a3ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:39.156535 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d97654-73c4-4271-92a6-1da42d18a3ce admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:39.160274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0d852-23d1-4a73-9464-0fe3a9654ea2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:39.160976 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0d852-23d1-4a73-9464-0fe3a9654ea2 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:39.161297 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0d852-23d1-4a73-9464-0fe3a9654ea2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:39.161581 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0d852-23d1-4a73-9464-0fe3a9654ea2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:39.174525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0d852-23d1-4a73-9464-0fe3a9654ea2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:39.174525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0d852-23d1-4a73-9464-0fe3a9654ea2 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:39.185021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1064/2124] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:39.491502 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:39.733112 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:39.788353 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954107d-2021-4df3-a9bf-05716035c9d3 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:39.880591 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954107d-2021-4df3-a9bf-05716035c9d3 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:39.885904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0bc66-aa38-4f40-bdaa-fa868f2712b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:39.886558 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0bc66-aa38-4f40-bdaa-fa868f2712b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:39.886852 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0bc66-aa38-4f40-bdaa-fa868f2712b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:39.887115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0bc66-aa38-4f40-bdaa-fa868f2712b6 admin admin] PATH_INFO: `/v3/users/1728e7e5f236414ca5ec39e99004a8be` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:39.891057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0bc66-aa38-4f40-bdaa-fa868f2712b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1728e7e5f236414ca5ec39e99004a8be)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:39.891365 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0bc66-aa38-4f40-bdaa-fa868f2712b6 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:40.141909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-90f0bc66-aa38-4f40-bdaa-fa868f2712b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1728e7e5f236414ca5ec39e99004a8be'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:40.150722 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:40.165443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f2a9e-0365-467b-91c7-1390b139d189 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:40.165727 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f2a9e-0365-467b-91c7-1390b139d189 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:40.166004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f2a9e-0365-467b-91c7-1390b139d189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:40.205925 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-3a6f2a9e-0365-467b-91c7-1390b139d189 None None] MFA Rules not processed for user `37554a0aebaf4dd486aeeb4ce830d48a`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:40.207330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3a6f2a9e-0365-467b-91c7-1390b139d189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:40.268318 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-90f0bc66-aa38-4f40-bdaa-fa868f2712b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1728e7e5f236414ca5ec39e99004a8be'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:40.269876 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-90f0bc66-aa38-4f40-bdaa-fa868f2712b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1728e7e5f236414ca5ec39e99004a8be'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:40.270590 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-90f0bc66-aa38-4f40-bdaa-fa868f2712b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1728e7e5f236414ca5ec39e99004a8be'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:40.286740 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-90f0bc66-aa38-4f40-bdaa-fa868f2712b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1728e7e5f236414ca5ec39e99004a8be'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:40.315586 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1065/2125] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:40.324462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1061/2126] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:39 2022] DELETE /identity/v3/users/1728e7e5f236414ca5ec39e99004a8be => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:40.329942 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f2a9e-0365-467b-91c7-1390b139d189 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:40.329942 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3a6f2a9e-0365-467b-91c7-1390b139d189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:40.331321 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6e54c-944b-4385-ae3d-d40671bdaab8 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:40.331611 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6e54c-944b-4385-ae3d-d40671bdaab8 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:40.331891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6e54c-944b-4385-ae3d-d40671bdaab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:40.344354 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f2a9e-0365-467b-91c7-1390b139d189 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:40.353396 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-67c6e54c-944b-4385-ae3d-d40671bdaab8 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:40.354745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-67c6e54c-944b-4385-ae3d-d40671bdaab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:40.394961 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:40.407094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59c5f4-4d97-48de-b3b8-688ddcd828c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:40.407705 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59c5f4-4d97-48de-b3b8-688ddcd828c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:40.407982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59c5f4-4d97-48de-b3b8-688ddcd828c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:40.408259 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59c5f4-4d97-48de-b3b8-688ddcd828c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:40.410115 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5a59c5f4-4d97-48de-b3b8-688ddcd828c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:40.427578 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1062/2127] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:40.442254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6e54c-944b-4385-ae3d-d40671bdaab8 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:40.443949 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-67c6e54c-944b-4385-ae3d-d40671bdaab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:40.479684 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59c5f4-4d97-48de-b3b8-688ddcd828c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:40.481199 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59c5f4-4d97-48de-b3b8-688ddcd828c1 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:40.497696 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1066/2128] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:40.511036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6e54c-944b-4385-ae3d-d40671bdaab8 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:40.515209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c339437-5c47-437d-ba51-7a927accd013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:40.515850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c339437-5c47-437d-ba51-7a927accd013 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:40.516129 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c339437-5c47-437d-ba51-7a927accd013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:40.516406 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c339437-5c47-437d-ba51-7a927accd013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:40.526966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c339437-5c47-437d-ba51-7a927accd013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:40.528556 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c339437-5c47-437d-ba51-7a927accd013 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:40.538909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1063/2129] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:37:40.749794 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:40.989532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59c5f4-4d97-48de-b3b8-688ddcd828c1 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:40.992471 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5a59c5f4-4d97-48de-b3b8-688ddcd828c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:41.019092 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59c5f4-4d97-48de-b3b8-688ddcd828c1 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:41.027528 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26520166-b07c-4d56-8526-2a088351d7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:41.028179 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26520166-b07c-4d56-8526-2a088351d7cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:41.028459 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26520166-b07c-4d56-8526-2a088351d7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:41.028737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26520166-b07c-4d56-8526-2a088351d7cb admin admin] PATH_INFO: `/v3/projects/a2317296ebf349869d03736c7434396a` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:41.029943 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26520166-b07c-4d56-8526-2a088351d7cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2317296ebf349869d03736c7434396a)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:41.032150 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26520166-b07c-4d56-8526-2a088351d7cb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:41.191400 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-26520166-b07c-4d56-8526-2a088351d7cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2317296ebf349869d03736c7434396a', 'request_id': 'req-26520166-b07c-4d56-8526-2a088351d7cb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:41.200418 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-26520166-b07c-4d56-8526-2a088351d7cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2317296ebf349869d03736c7434396a', 'request_id': 'req-26520166-b07c-4d56-8526-2a088351d7cb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:41.207803 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-26520166-b07c-4d56-8526-2a088351d7cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2317296ebf349869d03736c7434396a', 'request_id': 'req-26520166-b07c-4d56-8526-2a088351d7cb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:41.275894 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:41.311265 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c339437-5c47-437d-ba51-7a927accd013 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:41.313029 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-26520166-b07c-4d56-8526-2a088351d7cb admin admin] The token cache is being invalidate because project a2317296ebf349869d03736c7434396a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:41.313354 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-26520166-b07c-4d56-8526-2a088351d7cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:41.316148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-26520166-b07c-4d56-8526-2a088351d7cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:41.322904 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1067/2130] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:40 2022] DELETE /identity/v3/projects/a2317296ebf349869d03736c7434396a => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:41.328313 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c339437-5c47-437d-ba51-7a927accd013 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:41.328635 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26520166-b07c-4d56-8526-2a088351d7cb admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:41.330750 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-26520166-b07c-4d56-8526-2a088351d7cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:41.407100 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd97a8-84b5-49c1-81f6-ceefa21e59af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:41.408261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd97a8-84b5-49c1-81f6-ceefa21e59af service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:41.408774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd97a8-84b5-49c1-81f6-ceefa21e59af service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:41.409527 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd97a8-84b5-49c1-81f6-ceefa21e59af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:41.413325 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-1abd97a8-84b5-49c1-81f6-ceefa21e59af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:41.426723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26520166-b07c-4d56-8526-2a088351d7cb admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:41.432548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a77d2-ecbc-495b-876c-2b69f3b6b5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:41.433834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a77d2-ecbc-495b-876c-2b69f3b6b5e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:41.434309 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a77d2-ecbc-495b-876c-2b69f3b6b5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:41.434789 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a77d2-ecbc-495b-876c-2b69f3b6b5e2 admin admin] PATH_INFO: `/v3/users/a4f712a8587f4ac0a3f5a31ccac44683` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:41.436596 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a77d2-ecbc-495b-876c-2b69f3b6b5e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4f712a8587f4ac0a3f5a31ccac44683)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:41.439241 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a77d2-ecbc-495b-876c-2b69f3b6b5e2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:41.492092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd97a8-84b5-49c1-81f6-ceefa21e59af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:41.494622 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd97a8-84b5-49c1-81f6-ceefa21e59af service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:41.510917 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1064/2131] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:41.689152 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3c7a77d2-ecbc-495b-876c-2b69f3b6b5e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4f712a8587f4ac0a3f5a31ccac44683'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:41.689807 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3c7a77d2-ecbc-495b-876c-2b69f3b6b5e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4f712a8587f4ac0a3f5a31ccac44683'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:41.756511 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3c7a77d2-ecbc-495b-876c-2b69f3b6b5e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4f712a8587f4ac0a3f5a31ccac44683'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:41.757223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3c7a77d2-ecbc-495b-876c-2b69f3b6b5e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4f712a8587f4ac0a3f5a31ccac44683'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:41.772170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3c7a77d2-ecbc-495b-876c-2b69f3b6b5e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4f712a8587f4ac0a3f5a31ccac44683'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:41.785510 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1068/2132] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:41 2022] DELETE /identity/v3/users/a4f712a8587f4ac0a3f5a31ccac44683 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:41.984635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:42.128152 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:42.246676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd97a8-84b5-49c1-81f6-ceefa21e59af service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:42.261285 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd97a8-84b5-49c1-81f6-ceefa21e59af service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:42.329974 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0488d-175c-4bd9-92e3-aac1b41081c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:42.330614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0488d-175c-4bd9-92e3-aac1b41081c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:42.330921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0488d-175c-4bd9-92e3-aac1b41081c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:42.331197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0488d-175c-4bd9-92e3-aac1b41081c0 admin admin] PATH_INFO: `/v3/projects/4fa7755b3fa14b99bcdcd4ac17c476ef` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:42.333925 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0488d-175c-4bd9-92e3-aac1b41081c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fa7755b3fa14b99bcdcd4ac17c476ef)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:42.337141 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0488d-175c-4bd9-92e3-aac1b41081c0 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:42.484839 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-baf0488d-175c-4bd9-92e3-aac1b41081c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fa7755b3fa14b99bcdcd4ac17c476ef', 'request_id': 'req-baf0488d-175c-4bd9-92e3-aac1b41081c0'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:42.490345 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-baf0488d-175c-4bd9-92e3-aac1b41081c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fa7755b3fa14b99bcdcd4ac17c476ef', 'request_id': 'req-baf0488d-175c-4bd9-92e3-aac1b41081c0'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:42.496783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-baf0488d-175c-4bd9-92e3-aac1b41081c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fa7755b3fa14b99bcdcd4ac17c476ef', 'request_id': 'req-baf0488d-175c-4bd9-92e3-aac1b41081c0'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:42.585213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-baf0488d-175c-4bd9-92e3-aac1b41081c0 admin admin] The token cache is being invalidate because project 4fa7755b3fa14b99bcdcd4ac17c476ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:42.585213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-baf0488d-175c-4bd9-92e3-aac1b41081c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:42.585213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-baf0488d-175c-4bd9-92e3-aac1b41081c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:42.587601 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1065/2133] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:42 2022] DELETE /identity/v3/projects/4fa7755b3fa14b99bcdcd4ac17c476ef => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:42.599990 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e6df9-f7e7-48df-bb3a-9973195ecfb3 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:42.604549 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e6df9-f7e7-48df-bb3a-9973195ecfb3 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:42.604841 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e6df9-f7e7-48df-bb3a-9973195ecfb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:42.615566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0488d-175c-4bd9-92e3-aac1b41081c0 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:42.619743 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-baf0488d-175c-4bd9-92e3-aac1b41081c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:42.632601 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-d35e6df9-f7e7-48df-bb3a-9973195ecfb3 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:42.634845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d35e6df9-f7e7-48df-bb3a-9973195ecfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:42.775546 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1069/2134] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:42.785816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b8c9d-50a5-4fa7-bcca-50d293748113 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:42.786100 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b8c9d-50a5-4fa7-bcca-50d293748113 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:42.786359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b8c9d-50a5-4fa7-bcca-50d293748113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:42.806733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0488d-175c-4bd9-92e3-aac1b41081c0 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:42.822991 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5b626-9e03-4c3a-96ee-54a5be7afab7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:42.823739 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5b626-9e03-4c3a-96ee-54a5be7afab7 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:42.824167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5b626-9e03-4c3a-96ee-54a5be7afab7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:42.824493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5b626-9e03-4c3a-96ee-54a5be7afab7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:42.829687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-a3e5b626-9e03-4c3a-96ee-54a5be7afab7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:42.852628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5c8b8c9d-50a5-4fa7-bcca-50d293748113 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:42.855035 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5c8b8c9d-50a5-4fa7-bcca-50d293748113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:42.865715 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1070/2135] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:42.873752 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b8c9d-50a5-4fa7-bcca-50d293748113 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:42.882199 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b8c9d-50a5-4fa7-bcca-50d293748113 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:42.889468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2f3d2-bd5a-4996-8e65-5fdda71bc997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:42.891446 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2f3d2-bd5a-4996-8e65-5fdda71bc997 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:42.892305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2f3d2-bd5a-4996-8e65-5fdda71bc997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:42.892953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2f3d2-bd5a-4996-8e65-5fdda71bc997 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:42.894169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2f3d2-bd5a-4996-8e65-5fdda71bc997 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:42.897901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2f3d2-bd5a-4996-8e65-5fdda71bc997 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:42.907009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2f3d2-bd5a-4996-8e65-5fdda71bc997 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:42.933233 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1071/2136] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:37:42.942764 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2f3d2-bd5a-4996-8e65-5fdda71bc997 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:42.959296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2f3d2-bd5a-4996-8e65-5fdda71bc997 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:42.959483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5b626-9e03-4c3a-96ee-54a5be7afab7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:42.961562 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5b626-9e03-4c3a-96ee-54a5be7afab7 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:42.967689 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3353e1e-ef30-4369-a469-bf8045ec05cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:42.969226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3353e1e-ef30-4369-a469-bf8045ec05cb admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:42.970107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3353e1e-ef30-4369-a469-bf8045ec05cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:42.970637 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3353e1e-ef30-4369-a469-bf8045ec05cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:42.973413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3353e1e-ef30-4369-a469-bf8045ec05cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:42.976065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3353e1e-ef30-4369-a469-bf8045ec05cb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:43.006615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1066/2137] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:43.110759 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1072/2138] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:42 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:43.113420 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5b626-9e03-4c3a-96ee-54a5be7afab7 service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:43.124555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5b626-9e03-4c3a-96ee-54a5be7afab7 service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:43.222447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3353e1e-ef30-4369-a469-bf8045ec05cb admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:43.224185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e3353e1e-ef30-4369-a469-bf8045ec05cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:43.241181 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0326c0-cbb1-4b90-b795-e75afb907319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:43.241810 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0326c0-cbb1-4b90-b795-e75afb907319 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:43.242685 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0326c0-cbb1-4b90-b795-e75afb907319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:43.242972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0326c0-cbb1-4b90-b795-e75afb907319 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:43.243624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0326c0-cbb1-4b90-b795-e75afb907319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:43.251295 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0326c0-cbb1-4b90-b795-e75afb907319 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:43.353862 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3353e1e-ef30-4369-a469-bf8045ec05cb admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:43.359948 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db542d4-7096-4db1-9e6a-d0ba776c5c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:43.360717 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db542d4-7096-4db1-9e6a-d0ba776c5c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:43.362020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db542d4-7096-4db1-9e6a-d0ba776c5c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:43.362020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db542d4-7096-4db1-9e6a-d0ba776c5c6f admin admin] PATH_INFO: `/v3/users/97a12c94c8884744a6c1f7f349e60ea1` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:43.363212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db542d4-7096-4db1-9e6a-d0ba776c5c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=97a12c94c8884744a6c1f7f349e60ea1)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:43.365792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db542d4-7096-4db1-9e6a-d0ba776c5c6f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:43.435231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1067/2139] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:43 2022] POST /identity/v3/users => generated 293 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:43.436131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0326c0-cbb1-4b90-b795-e75afb907319 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:43.452498 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0326c0-cbb1-4b90-b795-e75afb907319 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:43.457354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60f814-7521-455c-8705-c9de075c5098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:43.457972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60f814-7521-455c-8705-c9de075c5098 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:43.458233 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60f814-7521-455c-8705-c9de075c5098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:43.458500 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60f814-7521-455c-8705-c9de075c5098 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:43.459144 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60f814-7521-455c-8705-c9de075c5098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:43.460734 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60f814-7521-455c-8705-c9de075c5098 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:43.471750 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1068/2140] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:43.472757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60f814-7521-455c-8705-c9de075c5098 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:43.474438 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-9a60f814-7521-455c-8705-c9de075c5098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:43.534959 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60f814-7521-455c-8705-c9de075c5098 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:43.539361 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c2320-e4bb-4e49-a04b-365a62eea0f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:43.539976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c2320-e4bb-4e49-a04b-365a62eea0f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:43.540323 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c2320-e4bb-4e49-a04b-365a62eea0f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:43.540595 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c2320-e4bb-4e49-a04b-365a62eea0f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:43.542359 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4d3c2320-e4bb-4e49-a04b-365a62eea0f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:43.594898 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4db542d4-7096-4db1-9e6a-d0ba776c5c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97a12c94c8884744a6c1f7f349e60ea1'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:43.595919 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4db542d4-7096-4db1-9e6a-d0ba776c5c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97a12c94c8884744a6c1f7f349e60ea1'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:43.606246 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:43.609536 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c2320-e4bb-4e49-a04b-365a62eea0f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:43.611018 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c2320-e4bb-4e49-a04b-365a62eea0f9 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:43.621161 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1069/2141] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:43.621957 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c2320-e4bb-4e49-a04b-365a62eea0f9 service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:43.631735 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c2320-e4bb-4e49-a04b-365a62eea0f9 service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:43.636829 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405b969-f804-446d-b6f9-e946eff10078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:43.637456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405b969-f804-446d-b6f9-e946eff10078 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:43.640284 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405b969-f804-446d-b6f9-e946eff10078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:43.640284 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405b969-f804-446d-b6f9-e946eff10078 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:43.640284 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405b969-f804-446d-b6f9-e946eff10078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:43.640592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405b969-f804-446d-b6f9-e946eff10078 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:43.651567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1070/2142] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:43.652384 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405b969-f804-446d-b6f9-e946eff10078 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:43.654118 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-7405b969-f804-446d-b6f9-e946eff10078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:43.666622 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405b969-f804-446d-b6f9-e946eff10078 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:43.671351 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762fa37-15b4-41fb-8f46-7f62ac53f6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:43.671975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762fa37-15b4-41fb-8f46-7f62ac53f6d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:43.672251 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762fa37-15b4-41fb-8f46-7f62ac53f6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:43.672530 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762fa37-15b4-41fb-8f46-7f62ac53f6d5 admin admin] PATH_INFO: `/v3/users/5fecaefd115842fe80009d70921c3806` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:43.672978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4db542d4-7096-4db1-9e6a-d0ba776c5c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97a12c94c8884744a6c1f7f349e60ea1'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:43.673771 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4db542d4-7096-4db1-9e6a-d0ba776c5c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97a12c94c8884744a6c1f7f349e60ea1'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:43.673873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762fa37-15b4-41fb-8f46-7f62ac53f6d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fecaefd115842fe80009d70921c3806)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:43.675781 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762fa37-15b4-41fb-8f46-7f62ac53f6d5 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:43.687214 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4db542d4-7096-4db1-9e6a-d0ba776c5c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97a12c94c8884744a6c1f7f349e60ea1'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:43.700180 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1073/2143] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:43 2022] DELETE /identity/v3/users/97a12c94c8884744a6c1f7f349e60ea1 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:43.701151 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db542d4-7096-4db1-9e6a-d0ba776c5c6f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:43.702465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4db542d4-7096-4db1-9e6a-d0ba776c5c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:43.766216 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db542d4-7096-4db1-9e6a-d0ba776c5c6f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:43.770842 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfcb9f-6105-45cc-93e8-2334199146e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:43.771500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfcb9f-6105-45cc-93e8-2334199146e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:43.771784 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfcb9f-6105-45cc-93e8-2334199146e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:43.772064 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfcb9f-6105-45cc-93e8-2334199146e5 admin admin] PATH_INFO: `/v3/projects/40f0a0ad63d14445a7923ff277fd0b01/users/2ef59ded29b641b09aa2018622661f6b/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:43.786477 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfcb9f-6105-45cc-93e8-2334199146e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40f0a0ad63d14445a7923ff277fd0b01, user_id=2ef59ded29b641b09aa2018622661f6b, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:43.789234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfcb9f-6105-45cc-93e8-2334199146e5 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:43.875522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1074/2144] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:43 2022] PUT /identity/v3/projects/40f0a0ad63d14445a7923ff277fd0b01/users/2ef59ded29b641b09aa2018622661f6b/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:43.876668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85d51d-b8a8-46f9-a657-7c8a0b675a7e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:43.878162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85d51d-b8a8-46f9-a657-7c8a0b675a7e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:43.878162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85d51d-b8a8-46f9-a657-7c8a0b675a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:43.896734 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-eb85d51d-b8a8-46f9-a657-7c8a0b675a7e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:43.898001 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-eb85d51d-b8a8-46f9-a657-7c8a0b675a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:43.899864 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:43.913371 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f762fa37-15b4-41fb-8f46-7f62ac53f6d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fecaefd115842fe80009d70921c3806'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:43.959458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1075/2145] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:43.960723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce4f68-f8ae-4d22-97a0-17a21080460b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:43.961007 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce4f68-f8ae-4d22-97a0-17a21080460b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:43.961320 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce4f68-f8ae-4d22-97a0-17a21080460b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:43.971953 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f762fa37-15b4-41fb-8f46-7f62ac53f6d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fecaefd115842fe80009d70921c3806'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:43.972584 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f762fa37-15b4-41fb-8f46-7f62ac53f6d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fecaefd115842fe80009d70921c3806'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:43.973221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f762fa37-15b4-41fb-8f46-7f62ac53f6d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fecaefd115842fe80009d70921c3806'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:43.981044 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f762fa37-15b4-41fb-8f46-7f62ac53f6d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fecaefd115842fe80009d70921c3806'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:43.988691 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f7ce4f68-f8ae-4d22-97a0-17a21080460b None None] MFA Rules not processed for user `2ef59ded29b641b09aa2018622661f6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:43.989938 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f7ce4f68-f8ae-4d22-97a0-17a21080460b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:43.999128 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1071/2146] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:43 2022] DELETE /identity/v3/users/5fecaefd115842fe80009d70921c3806 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:43.999941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762fa37-15b4-41fb-8f46-7f62ac53f6d5 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:44.001565 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-f762fa37-15b4-41fb-8f46-7f62ac53f6d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:44.064097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1076/2147] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:44.075246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762fa37-15b4-41fb-8f46-7f62ac53f6d5 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:44.076858 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce4f68-f8ae-4d22-97a0-17a21080460b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:44.078487 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f7ce4f68-f8ae-4d22-97a0-17a21080460b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:44.079350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c531b-467c-408c-8c64-01984295cba5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:44.079967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c531b-467c-408c-8c64-01984295cba5 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:44.080258 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c531b-467c-408c-8c64-01984295cba5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:44.080530 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c531b-467c-408c-8c64-01984295cba5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:44.082504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-171c531b-467c-408c-8c64-01984295cba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:44.145514 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce4f68-f8ae-4d22-97a0-17a21080460b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:44.156902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c531b-467c-408c-8c64-01984295cba5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:44.158395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c531b-467c-408c-8c64-01984295cba5 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:44.169394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:44.171281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1072/2148] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:44.203604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b0c59-bdc9-462e-813d-2237fdd2bd48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:44.204218 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b0c59-bdc9-462e-813d-2237fdd2bd48 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:44.204495 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b0c59-bdc9-462e-813d-2237fdd2bd48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:44.204766 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b0c59-bdc9-462e-813d-2237fdd2bd48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:44.260461 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b0c59-bdc9-462e-813d-2237fdd2bd48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:44.261930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b0c59-bdc9-462e-813d-2237fdd2bd48 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:44.271245 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1077/2149] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:44.278298 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c531b-467c-408c-8c64-01984295cba5 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:44.280525 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-171c531b-467c-408c-8c64-01984295cba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:44.294734 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b0c59-bdc9-462e-813d-2237fdd2bd48 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:44.298665 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8f4b0c59-bdc9-462e-813d-2237fdd2bd48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:44.302319 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c531b-467c-408c-8c64-01984295cba5 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:44.308383 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc0af9-41ec-4fd5-9a05-a6231ee59aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:44.309140 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc0af9-41ec-4fd5-9a05-a6231ee59aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:44.309491 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc0af9-41ec-4fd5-9a05-a6231ee59aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:44.309840 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc0af9-41ec-4fd5-9a05-a6231ee59aa8 admin admin] PATH_INFO: `/v3/projects/a48df5556fc247fba5eaf7d3d1ecf44f` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:44.311257 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc0af9-41ec-4fd5-9a05-a6231ee59aa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a48df5556fc247fba5eaf7d3d1ecf44f)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:44.314181 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc0af9-41ec-4fd5-9a05-a6231ee59aa8 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:44.333814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b0c59-bdc9-462e-813d-2237fdd2bd48 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:44.338892 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10e99b-dd59-4025-a7a5-e41c954d3928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:44.339556 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10e99b-dd59-4025-a7a5-e41c954d3928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:44.339834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10e99b-dd59-4025-a7a5-e41c954d3928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:44.340137 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10e99b-dd59-4025-a7a5-e41c954d3928 admin admin] PATH_INFO: `/v3/users/2ef59ded29b641b09aa2018622661f6b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:44.341580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10e99b-dd59-4025-a7a5-e41c954d3928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ef59ded29b641b09aa2018622661f6b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:44.344346 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10e99b-dd59-4025-a7a5-e41c954d3928 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:44.417816 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2fbc0af9-41ec-4fd5-9a05-a6231ee59aa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a48df5556fc247fba5eaf7d3d1ecf44f', 'request_id': 'req-2fbc0af9-41ec-4fd5-9a05-a6231ee59aa8'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:44.423699 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2fbc0af9-41ec-4fd5-9a05-a6231ee59aa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a48df5556fc247fba5eaf7d3d1ecf44f', 'request_id': 'req-2fbc0af9-41ec-4fd5-9a05-a6231ee59aa8'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:44.430844 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2fbc0af9-41ec-4fd5-9a05-a6231ee59aa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a48df5556fc247fba5eaf7d3d1ecf44f', 'request_id': 'req-2fbc0af9-41ec-4fd5-9a05-a6231ee59aa8'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:44.527080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2fbc0af9-41ec-4fd5-9a05-a6231ee59aa8 admin admin] The token cache is being invalidate because project a48df5556fc247fba5eaf7d3d1ecf44f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:44.527432 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2fbc0af9-41ec-4fd5-9a05-a6231ee59aa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:44.528099 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2fbc0af9-41ec-4fd5-9a05-a6231ee59aa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:44.529635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1073/2150] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:44 2022] DELETE /identity/v3/projects/a48df5556fc247fba5eaf7d3d1ecf44f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:44.534568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f04475-4c94-4c82-bb00-8f8ea29671e7 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:44.534853 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f04475-4c94-4c82-bb00-8f8ea29671e7 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:44.535131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f04475-4c94-4c82-bb00-8f8ea29671e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:44.563624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-18f04475-4c94-4c82-bb00-8f8ea29671e7 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:44.565096 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-18f04475-4c94-4c82-bb00-8f8ea29671e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:44.565940 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-cf10e99b-dd59-4025-a7a5-e41c954d3928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef59ded29b641b09aa2018622661f6b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:44.566699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-cf10e99b-dd59-4025-a7a5-e41c954d3928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef59ded29b641b09aa2018622661f6b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:44.641102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1074/2151] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:44.648877 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d473c5-a28c-45e9-9801-daffb43fae0e None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:44.649468 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d473c5-a28c-45e9-9801-daffb43fae0e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:44.649665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d473c5-a28c-45e9-9801-daffb43fae0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:44.652198 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-cf10e99b-dd59-4025-a7a5-e41c954d3928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef59ded29b641b09aa2018622661f6b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:44.653697 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-cf10e99b-dd59-4025-a7a5-e41c954d3928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef59ded29b641b09aa2018622661f6b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:44.677555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-cf10e99b-dd59-4025-a7a5-e41c954d3928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef59ded29b641b09aa2018622661f6b'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:44.683882 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-50d473c5-a28c-45e9-9801-daffb43fae0e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:44.685839 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-50d473c5-a28c-45e9-9801-daffb43fae0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:44.691579 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1078/2152] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:44 2022] DELETE /identity/v3/users/2ef59ded29b641b09aa2018622661f6b => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:44.694255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10e99b-dd59-4025-a7a5-e41c954d3928 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:44.694255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cf10e99b-dd59-4025-a7a5-e41c954d3928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:44.701520 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1075/2153] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:44.702712 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342a6e9-e855-478e-b456-62ecf9e0dbbc None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:44.702956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342a6e9-e855-478e-b456-62ecf9e0dbbc None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:44.703236 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342a6e9-e855-478e-b456-62ecf9e0dbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:44.724011 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-4342a6e9-e855-478e-b456-62ecf9e0dbbc None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:44.725348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4342a6e9-e855-478e-b456-62ecf9e0dbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:44.771953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10e99b-dd59-4025-a7a5-e41c954d3928 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:44.777989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b17ac1-d0d6-427d-86ae-dde42542bd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:44.778654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b17ac1-d0d6-427d-86ae-dde42542bd33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:44.778933 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b17ac1-d0d6-427d-86ae-dde42542bd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:44.779213 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b17ac1-d0d6-427d-86ae-dde42542bd33 admin admin] PATH_INFO: `/v3/projects/c449e87bba034a3abedcf9290e4652c8` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:44.780341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b17ac1-d0d6-427d-86ae-dde42542bd33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c449e87bba034a3abedcf9290e4652c8)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:44.782377 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b17ac1-d0d6-427d-86ae-dde42542bd33 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:44.788422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1076/2154] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:44.789942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342a6e9-e855-478e-b456-62ecf9e0dbbc None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:44.802452 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342a6e9-e855-478e-b456-62ecf9e0dbbc None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:44.807308 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5060c7-42b0-4fe4-9125-c689afe21fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:44.807957 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5060c7-42b0-4fe4-9125-c689afe21fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:44.808239 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5060c7-42b0-4fe4-9125-c689afe21fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:44.808513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5060c7-42b0-4fe4-9125-c689afe21fb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:44.809373 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5060c7-42b0-4fe4-9125-c689afe21fb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:44.810922 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5060c7-42b0-4fe4-9125-c689afe21fb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:44.811401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5060c7-42b0-4fe4-9125-c689afe21fb5 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:44.824375 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1077/2155] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:37:44.825283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5060c7-42b0-4fe4-9125-c689afe21fb5 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:44.827020 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-aa5060c7-42b0-4fe4-9125-c689afe21fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:44.886010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5060c7-42b0-4fe4-9125-c689afe21fb5 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:44.889659 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd84919-147f-4ef5-8d9d-ae06dbd1201d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:44.891350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd84919-147f-4ef5-8d9d-ae06dbd1201d service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:44.891350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd84919-147f-4ef5-8d9d-ae06dbd1201d service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:44.891350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd84919-147f-4ef5-8d9d-ae06dbd1201d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:44.901904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd84919-147f-4ef5-8d9d-ae06dbd1201d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:44.903521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd84919-147f-4ef5-8d9d-ae06dbd1201d service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:44.914443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1078/2156] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:44.915195 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd84919-147f-4ef5-8d9d-ae06dbd1201d service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:44.924035 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-96b17ac1-d0d6-427d-86ae-dde42542bd33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c449e87bba034a3abedcf9290e4652c8', 'request_id': 'req-96b17ac1-d0d6-427d-86ae-dde42542bd33'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:44.928621 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd84919-147f-4ef5-8d9d-ae06dbd1201d service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:44.935214 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-96b17ac1-d0d6-427d-86ae-dde42542bd33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c449e87bba034a3abedcf9290e4652c8', 'request_id': 'req-96b17ac1-d0d6-427d-86ae-dde42542bd33'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:44.944443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-96b17ac1-d0d6-427d-86ae-dde42542bd33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c449e87bba034a3abedcf9290e4652c8', 'request_id': 'req-96b17ac1-d0d6-427d-86ae-dde42542bd33'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:44.967284 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:45.003330 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db541474-1bd4-4dd1-bc79-e63e2e3d34ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:45.004217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db541474-1bd4-4dd1-bc79-e63e2e3d34ff admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:45.004500 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db541474-1bd4-4dd1-bc79-e63e2e3d34ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:45.004778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db541474-1bd4-4dd1-bc79-e63e2e3d34ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:45.005451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db541474-1bd4-4dd1-bc79-e63e2e3d34ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:45.007374 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db541474-1bd4-4dd1-bc79-e63e2e3d34ff admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:45.067543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-96b17ac1-d0d6-427d-86ae-dde42542bd33 admin admin] The token cache is being invalidate because project c449e87bba034a3abedcf9290e4652c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:45.067836 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-96b17ac1-d0d6-427d-86ae-dde42542bd33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:45.068468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-96b17ac1-d0d6-427d-86ae-dde42542bd33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:45.071607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1079/2157] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:44 2022] DELETE /identity/v3/projects/c449e87bba034a3abedcf9290e4652c8 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:45.076119 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce799ae-f66d-48a8-aa32-01b5e15812a7 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:45.076399 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce799ae-f66d-48a8-aa32-01b5e15812a7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:45.076673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce799ae-f66d-48a8-aa32-01b5e15812a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:45.096266 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5ce799ae-f66d-48a8-aa32-01b5e15812a7 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:45.098875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5ce799ae-f66d-48a8-aa32-01b5e15812a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:45.105460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1080/2158] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:45.112772 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acddbf4-6d97-4438-b074-ec533248ac23 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:45.113078 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acddbf4-6d97-4438-b074-ec533248ac23 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:45.113371 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acddbf4-6d97-4438-b074-ec533248ac23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:45.127370 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1079/2159] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:44 2022] POST /identity/v3/projects => generated 393 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:45.133467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1acddbf4-6d97-4438-b074-ec533248ac23 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:45.134770 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1acddbf4-6d97-4438-b074-ec533248ac23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:45.136179 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db541474-1bd4-4dd1-bc79-e63e2e3d34ff admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:45.137846 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-db541474-1bd4-4dd1-bc79-e63e2e3d34ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:45.237336 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1081/2160] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:45.241463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acddbf4-6d97-4438-b074-ec533248ac23 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:45.243111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db541474-1bd4-4dd1-bc79-e63e2e3d34ff admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:45.249847 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977ae01-ee78-4534-98b4-0fd56523e36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:45.250480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977ae01-ee78-4534-98b4-0fd56523e36f admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:45.250893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977ae01-ee78-4534-98b4-0fd56523e36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:45.251178 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977ae01-ee78-4534-98b4-0fd56523e36f admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:45.252301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b977ae01-ee78-4534-98b4-0fd56523e36f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:45.254107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acddbf4-6d97-4438-b074-ec533248ac23 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:45.254678 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b977ae01-ee78-4534-98b4-0fd56523e36f admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:45.271564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212887b-a6bb-43e9-9a97-ff406126a3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:45.272467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212887b-a6bb-43e9-9a97-ff406126a3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:45.272758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212887b-a6bb-43e9-9a97-ff406126a3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:45.273049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212887b-a6bb-43e9-9a97-ff406126a3c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:45.274293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212887b-a6bb-43e9-9a97-ff406126a3c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:45.274606 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212887b-a6bb-43e9-9a97-ff406126a3c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:45.276908 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212887b-a6bb-43e9-9a97-ff406126a3c6 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:45.291866 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1082/2161] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:37:45.296090 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212887b-a6bb-43e9-9a97-ff406126a3c6 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:45.306598 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212887b-a6bb-43e9-9a97-ff406126a3c6 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:45.311450 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349d68b-a61a-4b94-9485-1f46423836a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:45.312298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349d68b-a61a-4b94-9485-1f46423836a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:45.312586 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349d68b-a61a-4b94-9485-1f46423836a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:45.312866 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349d68b-a61a-4b94-9485-1f46423836a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:45.313546 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349d68b-a61a-4b94-9485-1f46423836a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:45.315781 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349d68b-a61a-4b94-9485-1f46423836a7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:45.364730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1080/2162] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:45 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:37:45.365946 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977ae01-ee78-4534-98b4-0fd56523e36f admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:45.376229 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977ae01-ee78-4534-98b4-0fd56523e36f admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:45.380950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1159c-b5c2-45f2-8248-676fc6795444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:45.381603 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1159c-b5c2-45f2-8248-676fc6795444 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:45.381885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1159c-b5c2-45f2-8248-676fc6795444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:45.382163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1159c-b5c2-45f2-8248-676fc6795444 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:45.382817 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1159c-b5c2-45f2-8248-676fc6795444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:45.384415 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1159c-b5c2-45f2-8248-676fc6795444 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:45.395203 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1081/2163] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:45.399506 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1159c-b5c2-45f2-8248-676fc6795444 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:45.415473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1159c-b5c2-45f2-8248-676fc6795444 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:45.418523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a1542-c1e6-44eb-b028-91416117a4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:45.419121 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a1542-c1e6-44eb-b028-91416117a4fa admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:45.419402 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a1542-c1e6-44eb-b028-91416117a4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:45.419676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a1542-c1e6-44eb-b028-91416117a4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:45.420315 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a1542-c1e6-44eb-b028-91416117a4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:45.422249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a1542-c1e6-44eb-b028-91416117a4fa admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:45.432546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1082/2164] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:45.433506 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a1542-c1e6-44eb-b028-91416117a4fa admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:45.437923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-104a1542-c1e6-44eb-b028-91416117a4fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:45.451908 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1083/2165] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:45 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:37:45.452881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a1542-c1e6-44eb-b028-91416117a4fa admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:45.454117 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349d68b-a61a-4b94-9485-1f46423836a7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:45.509085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade80eb9-75ec-4bb2-9dd4-1d83ff0eeb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:45.509732 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade80eb9-75ec-4bb2-9dd4-1d83ff0eeb5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:45.510011 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade80eb9-75ec-4bb2-9dd4-1d83ff0eeb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:45.510287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade80eb9-75ec-4bb2-9dd4-1d83ff0eeb5c admin admin] PATH_INFO: `/v3/projects/40f0a0ad63d14445a7923ff277fd0b01` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:45.511400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade80eb9-75ec-4bb2-9dd4-1d83ff0eeb5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=40f0a0ad63d14445a7923ff277fd0b01)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:45.513395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade80eb9-75ec-4bb2-9dd4-1d83ff0eeb5c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:45.517046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349d68b-a61a-4b94-9485-1f46423836a7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:45.521713 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166dbc07-fd6d-42de-85d9-19b49b34f094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:45.522336 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166dbc07-fd6d-42de-85d9-19b49b34f094 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:45.522612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166dbc07-fd6d-42de-85d9-19b49b34f094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:45.522879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166dbc07-fd6d-42de-85d9-19b49b34f094 admin admin] PATH_INFO: `/v3/projects/af388d6e63d641528e8f00de41c59490/users/d3235496c38249a388e4afc263eadbc3/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:45.541812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166dbc07-fd6d-42de-85d9-19b49b34f094 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af388d6e63d641528e8f00de41c59490, user_id=d3235496c38249a388e4afc263eadbc3, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:45.544635 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166dbc07-fd6d-42de-85d9-19b49b34f094 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:45.687528 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1084/2166] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:45 2022] PUT /identity/v3/projects/af388d6e63d641528e8f00de41c59490/users/d3235496c38249a388e4afc263eadbc3/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:45.689143 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166dbc07-fd6d-42de-85d9-19b49b34f094 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:45.700816 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ade80eb9-75ec-4bb2-9dd4-1d83ff0eeb5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40f0a0ad63d14445a7923ff277fd0b01', 'request_id': 'req-ade80eb9-75ec-4bb2-9dd4-1d83ff0eeb5c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:45.707775 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ade80eb9-75ec-4bb2-9dd4-1d83ff0eeb5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40f0a0ad63d14445a7923ff277fd0b01', 'request_id': 'req-ade80eb9-75ec-4bb2-9dd4-1d83ff0eeb5c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:45.716134 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ade80eb9-75ec-4bb2-9dd4-1d83ff0eeb5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40f0a0ad63d14445a7923ff277fd0b01', 'request_id': 'req-ade80eb9-75ec-4bb2-9dd4-1d83ff0eeb5c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:45.765408 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166dbc07-fd6d-42de-85d9-19b49b34f094 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:45.784830 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:45.809296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ade80eb9-75ec-4bb2-9dd4-1d83ff0eeb5c admin admin] The token cache is being invalidate because project 40f0a0ad63d14445a7923ff277fd0b01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:45.809595 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ade80eb9-75ec-4bb2-9dd4-1d83ff0eeb5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:45.810354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ade80eb9-75ec-4bb2-9dd4-1d83ff0eeb5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:45.811997 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1083/2167] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:45 2022] DELETE /identity/v3/projects/40f0a0ad63d14445a7923ff277fd0b01 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:45.813528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae180dc-2a93-4cdf-af8f-b868a92a7833 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:45.813898 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae180dc-2a93-4cdf-af8f-b868a92a7833 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:45.814176 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae180dc-2a93-4cdf-af8f-b868a92a7833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:45.823334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ad457-f6c9-4a88-81a7-e051497a7960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:45.823982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ad457-f6c9-4a88-81a7-e051497a7960 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:45.824266 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ad457-f6c9-4a88-81a7-e051497a7960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:45.824575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ad457-f6c9-4a88-81a7-e051497a7960 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:45.825260 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ad457-f6c9-4a88-81a7-e051497a7960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:45.827274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ad457-f6c9-4a88-81a7-e051497a7960 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:45.857729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-8ae180dc-2a93-4cdf-af8f-b868a92a7833 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:45.859050 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-8ae180dc-2a93-4cdf-af8f-b868a92a7833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:45.867071 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1084/2168] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:45.877354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae180dc-2a93-4cdf-af8f-b868a92a7833 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:45.878986 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-8ae180dc-2a93-4cdf-af8f-b868a92a7833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:45.943950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae180dc-2a93-4cdf-af8f-b868a92a7833 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:45.948306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc4502-037d-46dc-ae31-186329a26987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:45.948936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc4502-037d-46dc-ae31-186329a26987 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:45.949249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc4502-037d-46dc-ae31-186329a26987 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:45.949527 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc4502-037d-46dc-ae31-186329a26987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:45.963073 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbc4502-037d-46dc-ae31-186329a26987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:45.964550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbc4502-037d-46dc-ae31-186329a26987 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:45.968013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1085/2169] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:45 2022] POST /identity/v3/users => generated 287 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:45.971960 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ad457-f6c9-4a88-81a7-e051497a7960 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:45.973896 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-985ad457-f6c9-4a88-81a7-e051497a7960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:45.978939 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1085/2170] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:37:45.997643 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ad457-f6c9-4a88-81a7-e051497a7960 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:46.003116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a801af-e77a-4531-ac2c-cdf5717fb8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:46.004195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a801af-e77a-4531-ac2c-cdf5717fb8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:46.007095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a801af-e77a-4531-ac2c-cdf5717fb8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:46.007095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a801af-e77a-4531-ac2c-cdf5717fb8f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:46.007095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a801af-e77a-4531-ac2c-cdf5717fb8f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:46.010077 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a801af-e77a-4531-ac2c-cdf5717fb8f6 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:46.024983 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1086/2171] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:37:46.029186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc4502-037d-46dc-ae31-186329a26987 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:46.039258 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc4502-037d-46dc-ae31-186329a26987 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:46.044056 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b30e9d-f355-427e-a712-1965d5cef37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:46.044647 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b30e9d-f355-427e-a712-1965d5cef37f admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:46.044929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b30e9d-f355-427e-a712-1965d5cef37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:46.045250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b30e9d-f355-427e-a712-1965d5cef37f admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:46.045915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b30e9d-f355-427e-a712-1965d5cef37f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:46.047599 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b30e9d-f355-427e-a712-1965d5cef37f admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:46.058085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1086/2172] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:46.062143 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a801af-e77a-4531-ac2c-cdf5717fb8f6 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:46.072849 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a801af-e77a-4531-ac2c-cdf5717fb8f6 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:46.077514 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be93d75-c7e6-4786-a5b8-d339520acbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:46.078180 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be93d75-c7e6-4786-a5b8-d339520acbd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:46.078462 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be93d75-c7e6-4786-a5b8-d339520acbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:46.078747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be93d75-c7e6-4786-a5b8-d339520acbd2 admin admin] PATH_INFO: `/v3/projects/90c8dde9309d4f128f53cd6ece43ea52/users/29ae2fba5ddb4b30a7a8a3ecf10d28bb/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:46.093604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be93d75-c7e6-4786-a5b8-d339520acbd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c8dde9309d4f128f53cd6ece43ea52, user_id=29ae2fba5ddb4b30a7a8a3ecf10d28bb, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:46.096368 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be93d75-c7e6-4786-a5b8-d339520acbd2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:46.187588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1087/2173] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:46 2022] PUT /identity/v3/projects/90c8dde9309d4f128f53cd6ece43ea52/users/29ae2fba5ddb4b30a7a8a3ecf10d28bb/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:37:46.193032 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e5ff5-14ed-4817-a13b-476c8509f832 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:46.193494 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e5ff5-14ed-4817-a13b-476c8509f832 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:46.193641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e5ff5-14ed-4817-a13b-476c8509f832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:46.213184 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-986e5ff5-14ed-4817-a13b-476c8509f832 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:46.214376 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-986e5ff5-14ed-4817-a13b-476c8509f832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:46.272383 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1087/2174] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:46.279012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:46.283327 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be93d75-c7e6-4786-a5b8-d339520acbd2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:46.285000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1be93d75-c7e6-4786-a5b8-d339520acbd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:46.362384 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be93d75-c7e6-4786-a5b8-d339520acbd2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:46.366321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656a908-0801-4516-b173-dbeafa8c6a54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:46.368212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656a908-0801-4516-b173-dbeafa8c6a54 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:46.368212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656a908-0801-4516-b173-dbeafa8c6a54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:46.368212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656a908-0801-4516-b173-dbeafa8c6a54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:46.377336 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656a908-0801-4516-b173-dbeafa8c6a54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:46.378768 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656a908-0801-4516-b173-dbeafa8c6a54 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:46.389208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1088/2175] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:46.462610 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:47.613047 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:48.019306 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1b90d-e85c-4354-b4f2-199c03b3fe71 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:48.019564 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1b90d-e85c-4354-b4f2-199c03b3fe71 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:48.019931 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1b90d-e85c-4354-b4f2-199c03b3fe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:48.033088 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:48.078780 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-6df1b90d-e85c-4354-b4f2-199c03b3fe71 None None] MFA Rules not processed for user `29ae2fba5ddb4b30a7a8a3ecf10d28bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:48.080362 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-6df1b90d-e85c-4354-b4f2-199c03b3fe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:48.166467 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1088/2176] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:48.177812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656a908-0801-4516-b173-dbeafa8c6a54 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:48.179542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8656a908-0801-4516-b173-dbeafa8c6a54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:48.246475 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656a908-0801-4516-b173-dbeafa8c6a54 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:48.251114 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b49b0e-7410-47f3-9ccf-e5ee6be23c89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:48.251739 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b49b0e-7410-47f3-9ccf-e5ee6be23c89 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:48.252016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b49b0e-7410-47f3-9ccf-e5ee6be23c89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:48.252324 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b49b0e-7410-47f3-9ccf-e5ee6be23c89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:48.262376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b49b0e-7410-47f3-9ccf-e5ee6be23c89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:48.264060 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b49b0e-7410-47f3-9ccf-e5ee6be23c89 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:48.276671 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1089/2177] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:48.656987 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1b90d-e85c-4354-b4f2-199c03b3fe71 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:48.663231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-6df1b90d-e85c-4354-b4f2-199c03b3fe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:48.683880 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1b90d-e85c-4354-b4f2-199c03b3fe71 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:48.695472 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e847a-d16f-4c96-b469-06f350ec46df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:48.696483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e847a-d16f-4c96-b469-06f350ec46df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:48.697170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e847a-d16f-4c96-b469-06f350ec46df admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:48.697596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e847a-d16f-4c96-b469-06f350ec46df admin admin] PATH_INFO: `/v3/users/e3c892f5837340258eac011d84bcac22` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:48.699873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e847a-d16f-4c96-b469-06f350ec46df admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3c892f5837340258eac011d84bcac22)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:48.705292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e847a-d16f-4c96-b469-06f350ec46df admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:48.972409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-732e847a-d16f-4c96-b469-06f350ec46df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3c892f5837340258eac011d84bcac22'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:48.989251 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:49.052031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-732e847a-d16f-4c96-b469-06f350ec46df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3c892f5837340258eac011d84bcac22'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:49.052947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-732e847a-d16f-4c96-b469-06f350ec46df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3c892f5837340258eac011d84bcac22'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:49.053880 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-732e847a-d16f-4c96-b469-06f350ec46df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3c892f5837340258eac011d84bcac22'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:49.062917 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-732e847a-d16f-4c96-b469-06f350ec46df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3c892f5837340258eac011d84bcac22'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:49.082648 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1089/2178] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:48 2022] DELETE /identity/v3/users/e3c892f5837340258eac011d84bcac22 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:49.086953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56f42a-fa21-4727-b37c-5dfcc9d95b04 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:49.087451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56f42a-fa21-4727-b37c-5dfcc9d95b04 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:49.087531 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56f42a-fa21-4727-b37c-5dfcc9d95b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:49.109630 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-cb56f42a-fa21-4727-b37c-5dfcc9d95b04 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:49.110876 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cb56f42a-fa21-4727-b37c-5dfcc9d95b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:49.173525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1090/2179] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:49.183151 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e847a-d16f-4c96-b469-06f350ec46df admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:49.184718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-732e847a-d16f-4c96-b469-06f350ec46df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:49.246734 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e847a-d16f-4c96-b469-06f350ec46df admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:49.250584 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92c334-ef45-4654-af21-35aace021231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:49.251185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92c334-ef45-4654-af21-35aace021231 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:49.251460 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92c334-ef45-4654-af21-35aace021231 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:49.251729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92c334-ef45-4654-af21-35aace021231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:49.261022 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92c334-ef45-4654-af21-35aace021231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:49.262491 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92c334-ef45-4654-af21-35aace021231 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:49.274125 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1090/2180] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:49.576192 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:49.802746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56f42a-fa21-4727-b37c-5dfcc9d95b04 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:49.805531 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cb56f42a-fa21-4727-b37c-5dfcc9d95b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:49.836561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56f42a-fa21-4727-b37c-5dfcc9d95b04 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:49.845680 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16dedc-4814-426f-91f0-2748cfea7627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:49.846340 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16dedc-4814-426f-91f0-2748cfea7627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:49.846615 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16dedc-4814-426f-91f0-2748cfea7627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:49.846886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16dedc-4814-426f-91f0-2748cfea7627 admin admin] PATH_INFO: `/v3/projects/01fa72bebe0f4f8283a98e3553e997e3` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:49.848033 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16dedc-4814-426f-91f0-2748cfea7627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01fa72bebe0f4f8283a98e3553e997e3)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:49.854409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16dedc-4814-426f-91f0-2748cfea7627 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:49.994076 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9d16dedc-4814-426f-91f0-2748cfea7627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01fa72bebe0f4f8283a98e3553e997e3', 'request_id': 'req-9d16dedc-4814-426f-91f0-2748cfea7627'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:50.000914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9d16dedc-4814-426f-91f0-2748cfea7627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01fa72bebe0f4f8283a98e3553e997e3', 'request_id': 'req-9d16dedc-4814-426f-91f0-2748cfea7627'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:50.012223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9d16dedc-4814-426f-91f0-2748cfea7627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01fa72bebe0f4f8283a98e3553e997e3', 'request_id': 'req-9d16dedc-4814-426f-91f0-2748cfea7627'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:50.135372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9d16dedc-4814-426f-91f0-2748cfea7627 admin admin] The token cache is being invalidate because project 01fa72bebe0f4f8283a98e3553e997e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:50.137043 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9d16dedc-4814-426f-91f0-2748cfea7627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:50.138202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9d16dedc-4814-426f-91f0-2748cfea7627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:50.140039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebd466-e78c-466f-b60d-e89be4dcfcbb None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:50.140474 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1091/2181] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:49 2022] DELETE /identity/v3/projects/01fa72bebe0f4f8283a98e3553e997e3 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:50.140977 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebd466-e78c-466f-b60d-e89be4dcfcbb None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:50.141498 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebd466-e78c-466f-b60d-e89be4dcfcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:50.145601 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16dedc-4814-426f-91f0-2748cfea7627 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:50.149417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9d16dedc-4814-426f-91f0-2748cfea7627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:50.178174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-adebd466-e78c-466f-b60d-e89be4dcfcbb None None] MFA Rules not processed for user `29ae2fba5ddb4b30a7a8a3ecf10d28bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:50.181354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-adebd466-e78c-466f-b60d-e89be4dcfcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:50.245305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1091/2182] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:50.302276 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16dedc-4814-426f-91f0-2748cfea7627 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:50.312905 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaadc46-19aa-4f28-b1db-37981a7b1c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:50.313583 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaadc46-19aa-4f28-b1db-37981a7b1c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:50.313865 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaadc46-19aa-4f28-b1db-37981a7b1c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:50.314142 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaadc46-19aa-4f28-b1db-37981a7b1c9f admin admin] PATH_INFO: `/v3/users/c316f393e0fb45b785cd65b069942808` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:50.319221 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaadc46-19aa-4f28-b1db-37981a7b1c9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c316f393e0fb45b785cd65b069942808)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:50.321817 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaadc46-19aa-4f28-b1db-37981a7b1c9f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:50.843821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1eaadc46-19aa-4f28-b1db-37981a7b1c9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c316f393e0fb45b785cd65b069942808'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:50.844570 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1eaadc46-19aa-4f28-b1db-37981a7b1c9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c316f393e0fb45b785cd65b069942808'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:50.974440 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1eaadc46-19aa-4f28-b1db-37981a7b1c9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c316f393e0fb45b785cd65b069942808'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:50.975251 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1eaadc46-19aa-4f28-b1db-37981a7b1c9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c316f393e0fb45b785cd65b069942808'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:51.016983 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-1eaadc46-19aa-4f28-b1db-37981a7b1c9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c316f393e0fb45b785cd65b069942808'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:51.048266 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1092/2183] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:50 2022] DELETE /identity/v3/users/c316f393e0fb45b785cd65b069942808 => generated 0 bytes in 890 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:51.806755 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebd466-e78c-466f-b60d-e89be4dcfcbb None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:51.819468 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebd466-e78c-466f-b60d-e89be4dcfcbb None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:51.884769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aef389-ca00-4e14-a2bf-0c7a9d65af05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:51.885521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aef389-ca00-4e14-a2bf-0c7a9d65af05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:51.885830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aef389-ca00-4e14-a2bf-0c7a9d65af05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:51.886087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aef389-ca00-4e14-a2bf-0c7a9d65af05 admin admin] PATH_INFO: `/v3/projects/75f73f1c70b5444b88f81bf5f3868f13` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:51.887495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aef389-ca00-4e14-a2bf-0c7a9d65af05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f73f1c70b5444b88f81bf5f3868f13)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:51.889725 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aef389-ca00-4e14-a2bf-0c7a9d65af05 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:51.901748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:52.148652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-90aef389-ca00-4e14-a2bf-0c7a9d65af05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f73f1c70b5444b88f81bf5f3868f13', 'request_id': 'req-90aef389-ca00-4e14-a2bf-0c7a9d65af05'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:52.162748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-90aef389-ca00-4e14-a2bf-0c7a9d65af05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f73f1c70b5444b88f81bf5f3868f13', 'request_id': 'req-90aef389-ca00-4e14-a2bf-0c7a9d65af05'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:52.173540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-90aef389-ca00-4e14-a2bf-0c7a9d65af05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f73f1c70b5444b88f81bf5f3868f13', 'request_id': 'req-90aef389-ca00-4e14-a2bf-0c7a9d65af05'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:52.258957 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-90aef389-ca00-4e14-a2bf-0c7a9d65af05 admin admin] The token cache is being invalidate because project 75f73f1c70b5444b88f81bf5f3868f13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:52.259560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-90aef389-ca00-4e14-a2bf-0c7a9d65af05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:52.260720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-90aef389-ca00-4e14-a2bf-0c7a9d65af05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:52.262727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1092/2184] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:51 2022] DELETE /identity/v3/projects/75f73f1c70b5444b88f81bf5f3868f13 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:52.267326 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaadc46-19aa-4f28-b1db-37981a7b1c9f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:52.268895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1eaadc46-19aa-4f28-b1db-37981a7b1c9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:52.287458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaadc46-19aa-4f28-b1db-37981a7b1c9f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:52.365037 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2673ac-0bb7-43e8-8fa0-64e5c771847c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:52.365730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2673ac-0bb7-43e8-8fa0-64e5c771847c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:52.366273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2673ac-0bb7-43e8-8fa0-64e5c771847c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:52.367095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2673ac-0bb7-43e8-8fa0-64e5c771847c admin admin] PATH_INFO: `/v3/users/59b2fe7773744929aac82cf681e45345` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:52.368746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2673ac-0bb7-43e8-8fa0-64e5c771847c admin admin] RBAC: Authorizing `identity:delete_user(user_id=59b2fe7773744929aac82cf681e45345)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:52.371237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2673ac-0bb7-43e8-8fa0-64e5c771847c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:52.436734 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c6575-248d-4ecb-b8f8-98d757ef07f1 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:52.437007 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c6575-248d-4ecb-b8f8-98d757ef07f1 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:52.437545 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c6575-248d-4ecb-b8f8-98d757ef07f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:52.468006 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-5e8c6575-248d-4ecb-b8f8-98d757ef07f1 None None] MFA Rules not processed for user `d3235496c38249a388e4afc263eadbc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:52.469290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5e8c6575-248d-4ecb-b8f8-98d757ef07f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:52.545461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1093/2185] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:52.557516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a90dd-93ae-4d74-91b2-4d89dada744d None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:52.557691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a90dd-93ae-4d74-91b2-4d89dada744d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:52.557851 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a90dd-93ae-4d74-91b2-4d89dada744d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:52.593357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-2c9a90dd-93ae-4d74-91b2-4d89dada744d None None] MFA Rules not processed for user `d3235496c38249a388e4afc263eadbc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:52.593357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-2c9a90dd-93ae-4d74-91b2-4d89dada744d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:52.598292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1094/2186] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:52.609923 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a90dd-93ae-4d74-91b2-4d89dada744d None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:52.611305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-2c9a90dd-93ae-4d74-91b2-4d89dada744d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:52.699265 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a90dd-93ae-4d74-91b2-4d89dada744d None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:52.714108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71346f8-2e39-48d5-b346-a7f457ec785e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:52.715951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71346f8-2e39-48d5-b346-a7f457ec785e service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:52.716763 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71346f8-2e39-48d5-b346-a7f457ec785e service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:52.717438 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71346f8-2e39-48d5-b346-a7f457ec785e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:52.719718 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6a2673ac-0bb7-43e8-8fa0-64e5c771847c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59b2fe7773744929aac82cf681e45345'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:52.720410 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6a2673ac-0bb7-43e8-8fa0-64e5c771847c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59b2fe7773744929aac82cf681e45345'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:52.727601 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71346f8-2e39-48d5-b346-a7f457ec785e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:52.729399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71346f8-2e39-48d5-b346-a7f457ec785e service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:52.738668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1095/2187] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:52 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:52.844148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6a2673ac-0bb7-43e8-8fa0-64e5c771847c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59b2fe7773744929aac82cf681e45345'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:52.844833 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6a2673ac-0bb7-43e8-8fa0-64e5c771847c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59b2fe7773744929aac82cf681e45345'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:52.860739 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6a2673ac-0bb7-43e8-8fa0-64e5c771847c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59b2fe7773744929aac82cf681e45345'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:52.889878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1093/2188] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:37:52 2022] DELETE /identity/v3/users/59b2fe7773744929aac82cf681e45345 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:53.412025 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:53.522313 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71346f8-2e39-48d5-b346-a7f457ec785e service nova] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:53.523898 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-a71346f8-2e39-48d5-b346-a7f457ec785e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:53.541085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71346f8-2e39-48d5-b346-a7f457ec785e service nova] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:53.662882 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd2979-bfce-45bf-bc7e-ad5f090cd255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:53.664540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd2979-bfce-45bf-bc7e-ad5f090cd255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:53.666232 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd2979-bfce-45bf-bc7e-ad5f090cd255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:53.666704 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd2979-bfce-45bf-bc7e-ad5f090cd255 admin admin] PATH_INFO: `/v3/projects/e1a80d6abd504412a8aa993878390b8a` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:53.669716 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd2979-bfce-45bf-bc7e-ad5f090cd255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1a80d6abd504412a8aa993878390b8a)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:53.676046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a49735-4706-4e61-9d75-9a090b86fdce None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:53.676330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a49735-4706-4e61-9d75-9a090b86fdce None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:53.676762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a49735-4706-4e61-9d75-9a090b86fdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:53.676950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd2979-bfce-45bf-bc7e-ad5f090cd255 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:53.702754 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-46a49735-4706-4e61-9d75-9a090b86fdce None None] MFA Rules not processed for user `d3235496c38249a388e4afc263eadbc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:53.704351 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-46a49735-4706-4e61-9d75-9a090b86fdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:53.790885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1094/2189] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:37:53.856617 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-88cd2979-bfce-45bf-bc7e-ad5f090cd255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1a80d6abd504412a8aa993878390b8a', 'request_id': 'req-88cd2979-bfce-45bf-bc7e-ad5f090cd255'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:53.873937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-88cd2979-bfce-45bf-bc7e-ad5f090cd255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1a80d6abd504412a8aa993878390b8a', 'request_id': 'req-88cd2979-bfce-45bf-bc7e-ad5f090cd255'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:53.887309 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-88cd2979-bfce-45bf-bc7e-ad5f090cd255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1a80d6abd504412a8aa993878390b8a', 'request_id': 'req-88cd2979-bfce-45bf-bc7e-ad5f090cd255'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:53.994223 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-88cd2979-bfce-45bf-bc7e-ad5f090cd255 admin admin] The token cache is being invalidate because project e1a80d6abd504412a8aa993878390b8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:37:53.994428 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-88cd2979-bfce-45bf-bc7e-ad5f090cd255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:53.995639 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-88cd2979-bfce-45bf-bc7e-ad5f090cd255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:37:53.997906 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1096/2190] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:37:53 2022] DELETE /identity/v3/projects/e1a80d6abd504412a8aa993878390b8a => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:54.003626 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2976f1-9f95-4be6-8bb1-6be4763c87ac None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:54.003971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2976f1-9f95-4be6-8bb1-6be4763c87ac None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:54.004508 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2976f1-9f95-4be6-8bb1-6be4763c87ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:54.025274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8a2976f1-9f95-4be6-8bb1-6be4763c87ac None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:54.026802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8a2976f1-9f95-4be6-8bb1-6be4763c87ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:54.093410 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1095/2191] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:54.102472 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1aa180-1ea6-4bb7-a494-7ff0b8eeef25 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:54.102759 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1aa180-1ea6-4bb7-a494-7ff0b8eeef25 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:54.103657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1aa180-1ea6-4bb7-a494-7ff0b8eeef25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:54.125672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-7a1aa180-1ea6-4bb7-a494-7ff0b8eeef25 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:54.126470 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-7a1aa180-1ea6-4bb7-a494-7ff0b8eeef25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:54.135095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1097/2192] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:54.139653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2976f1-9f95-4be6-8bb1-6be4763c87ac None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:54.150072 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2976f1-9f95-4be6-8bb1-6be4763c87ac None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:54.155387 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde51e2c-d656-4b43-9bad-4b92a95fdd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:54.155997 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde51e2c-d656-4b43-9bad-4b92a95fdd8c admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:54.156275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde51e2c-d656-4b43-9bad-4b92a95fdd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:54.157567 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde51e2c-d656-4b43-9bad-4b92a95fdd8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:54.164466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde51e2c-d656-4b43-9bad-4b92a95fdd8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:37:54.168039 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde51e2c-d656-4b43-9bad-4b92a95fdd8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:54.169510 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde51e2c-d656-4b43-9bad-4b92a95fdd8c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:54.190187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1096/2193] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:37:54.196991 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1aa180-1ea6-4bb7-a494-7ff0b8eeef25 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:54.206092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1aa180-1ea6-4bb7-a494-7ff0b8eeef25 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:54.212976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340a7bf-40b9-4385-9b49-dbbc510c121f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:54.214037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340a7bf-40b9-4385-9b49-dbbc510c121f admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:54.214324 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340a7bf-40b9-4385-9b49-dbbc510c121f admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:54.214603 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340a7bf-40b9-4385-9b49-dbbc510c121f admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:54.215249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340a7bf-40b9-4385-9b49-dbbc510c121f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:54.219704 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340a7bf-40b9-4385-9b49-dbbc510c121f admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:54.325016 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1098/2194] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:37:54 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:37:54.329631 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde51e2c-d656-4b43-9bad-4b92a95fdd8c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:54.342616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde51e2c-d656-4b43-9bad-4b92a95fdd8c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:54.433826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4ada4-139f-42ed-82b3-0d4793f75612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:54.434549 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4ada4-139f-42ed-82b3-0d4793f75612 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:54.434872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4ada4-139f-42ed-82b3-0d4793f75612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:54.435150 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4ada4-139f-42ed-82b3-0d4793f75612 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:54.435833 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4ada4-139f-42ed-82b3-0d4793f75612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:54.437847 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4ada4-139f-42ed-82b3-0d4793f75612 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:54.544777 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1097/2195] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:37:54 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:37:54.552934 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340a7bf-40b9-4385-9b49-dbbc510c121f admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:54.574713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340a7bf-40b9-4385-9b49-dbbc510c121f admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:54.580725 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a430fc-ca4d-43ad-9948-fe1449c37e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:54.583860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a430fc-ca4d-43ad-9948-fe1449c37e39 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:54.583860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a430fc-ca4d-43ad-9948-fe1449c37e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:54.583860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a430fc-ca4d-43ad-9948-fe1449c37e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:54.583860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a430fc-ca4d-43ad-9948-fe1449c37e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:54.586541 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a430fc-ca4d-43ad-9948-fe1449c37e39 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:54.600883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1099/2196] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:54.605292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4ada4-139f-42ed-82b3-0d4793f75612 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:54.628299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4ada4-139f-42ed-82b3-0d4793f75612 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:54.652235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abfc5b-1ba1-49e9-b9ea-c7c2a3816ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:54.653595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abfc5b-1ba1-49e9-b9ea-c7c2a3816ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:54.654817 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abfc5b-1ba1-49e9-b9ea-c7c2a3816ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:54.655313 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abfc5b-1ba1-49e9-b9ea-c7c2a3816ab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:54.656420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73abfc5b-1ba1-49e9-b9ea-c7c2a3816ab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:54.658571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73abfc5b-1ba1-49e9-b9ea-c7c2a3816ab8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:54.673883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1098/2197] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:37:54.680139 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a430fc-ca4d-43ad-9948-fe1449c37e39 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:54.744160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a430fc-ca4d-43ad-9948-fe1449c37e39 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:54.768496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eaa379-c985-4696-a073-048ec18bcb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:54.769347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eaa379-c985-4696-a073-048ec18bcb0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:54.770074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eaa379-c985-4696-a073-048ec18bcb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:54.770360 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eaa379-c985-4696-a073-048ec18bcb0e admin admin] PATH_INFO: `/v3/projects/c84c95680abc4f28b49cd092ca96fe32/users/0b3fcdf3a12147f591c0ea67b57c8c7e/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:54.809313 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eaa379-c985-4696-a073-048ec18bcb0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84c95680abc4f28b49cd092ca96fe32, user_id=0b3fcdf3a12147f591c0ea67b57c8c7e, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:54.812058 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eaa379-c985-4696-a073-048ec18bcb0e admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:54.903246 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1100/2198] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:37:54 2022] PUT /identity/v3/projects/c84c95680abc4f28b49cd092ca96fe32/users/0b3fcdf3a12147f591c0ea67b57c8c7e/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:37:54.909571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf3320-225a-4f92-ab54-9dde29ec26b2 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:54.909889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf3320-225a-4f92-ab54-9dde29ec26b2 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:54.910170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf3320-225a-4f92-ab54-9dde29ec26b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:54.967470 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e7bf3320-225a-4f92-ab54-9dde29ec26b2 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:37:54.974975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e7bf3320-225a-4f92-ab54-9dde29ec26b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:55.095967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1099/2199] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:37:55.109598 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eaa379-c985-4696-a073-048ec18bcb0e admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:37:55.111441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e2eaa379-c985-4696-a073-048ec18bcb0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:37:55.211700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eaa379-c985-4696-a073-048ec18bcb0e admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:37:55.217672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074dd8c-3e04-41fd-ab57-8d46fff1b6a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:37:55.218422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074dd8c-3e04-41fd-ab57-8d46fff1b6a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:37:55.218801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074dd8c-3e04-41fd-ab57-8d46fff1b6a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:37:55.219313 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074dd8c-3e04-41fd-ab57-8d46fff1b6a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:37:55.235738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074dd8c-3e04-41fd-ab57-8d46fff1b6a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:37:55.237553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074dd8c-3e04-41fd-ab57-8d46fff1b6a9 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:37:55.251785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1101/2200] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:37:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:37:56.160982 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:57.600712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:59.131842 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:37:59.686957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:00.103321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ee831-9047-4ce1-903d-88202f86ce38 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:00.103606 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ee831-9047-4ce1-903d-88202f86ce38 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:00.103881 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ee831-9047-4ce1-903d-88202f86ce38 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:00.105240 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1100/2201] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 19 10:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:38:00.127427 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acf892-13ae-42a3-ac98-7e20307a010b None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:00.127710 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acf892-13ae-42a3-ac98-7e20307a010b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:00.127978 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acf892-13ae-42a3-ac98-7e20307a010b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:00.158838 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-07acf892-13ae-42a3-ac98-7e20307a010b None None] MFA Rules not processed for user `ce0ef47a60f54ca1853fd267e308c58b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:38:00.163180 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-07acf892-13ae-42a3-ac98-7e20307a010b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:00.262466 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1102/2202] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 19 10:38:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:38:00.269689 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f85476-9270-467b-97e9-828bbb94c545 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:00.269977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f85476-9270-467b-97e9-828bbb94c545 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:00.270255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f85476-9270-467b-97e9-828bbb94c545 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:00.271590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1101/2203] 10.222.0.42 () {54 vars in 961 bytes} [Mon Sep 19 10:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:38:00.278466 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc39beb-d8b8-48f7-9551-e00a81e7e027 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:00.285667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc39beb-d8b8-48f7-9551-e00a81e7e027 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:00.285667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc39beb-d8b8-48f7-9551-e00a81e7e027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:00.316537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-9cc39beb-d8b8-48f7-9551-e00a81e7e027 None None] MFA Rules not processed for user `ce0ef47a60f54ca1853fd267e308c58b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:38:00.318117 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-9cc39beb-d8b8-48f7-9551-e00a81e7e027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:00.328504 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1103/2204] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Sep 19 10:38:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:38:00.336901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f85476-9270-467b-97e9-828bbb94c545 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:00.359436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f85476-9270-467b-97e9-828bbb94c545 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:00.362263 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48b2e4-cfdf-41bb-a66c-ee93d4509fd3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0ef47a60f54ca1853fd267e308c58b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:00.363010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48b2e4-cfdf-41bb-a66c-ee93d4509fd3 service placement] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:00.363287 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48b2e4-cfdf-41bb-a66c-ee93d4509fd3 service placement] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:00.363556 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48b2e4-cfdf-41bb-a66c-ee93d4509fd3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:00.375929 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48b2e4-cfdf-41bb-a66c-ee93d4509fd3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:00.377624 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48b2e4-cfdf-41bb-a66c-ee93d4509fd3 service placement] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:00.418274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1102/2205] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:38:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:38:00.526052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:01.109157 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:01.463107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc39beb-d8b8-48f7-9551-e00a81e7e027 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:01.464522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-9cc39beb-d8b8-48f7-9551-e00a81e7e027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:01.507072 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc39beb-d8b8-48f7-9551-e00a81e7e027 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:01.519376 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e906c-8554-48e2-ae6d-3b832d9a8d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:01.520807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e906c-8554-48e2-ae6d-3b832d9a8d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:01.521275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e906c-8554-48e2-ae6d-3b832d9a8d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:01.522211 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e906c-8554-48e2-ae6d-3b832d9a8d3a admin admin] PATH_INFO: `/v3/users/9653b0c1ac9e42f498cf09ae6133bcbb` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:01.523582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e906c-8554-48e2-ae6d-3b832d9a8d3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9653b0c1ac9e42f498cf09ae6133bcbb)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:01.526599 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e906c-8554-48e2-ae6d-3b832d9a8d3a admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:01.911998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-0d6e906c-8554-48e2-ae6d-3b832d9a8d3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9653b0c1ac9e42f498cf09ae6133bcbb'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:01.999672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-0d6e906c-8554-48e2-ae6d-3b832d9a8d3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9653b0c1ac9e42f498cf09ae6133bcbb'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:02.000386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-0d6e906c-8554-48e2-ae6d-3b832d9a8d3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9653b0c1ac9e42f498cf09ae6133bcbb'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:02.001670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-0d6e906c-8554-48e2-ae6d-3b832d9a8d3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9653b0c1ac9e42f498cf09ae6133bcbb'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:02.016197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-0d6e906c-8554-48e2-ae6d-3b832d9a8d3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9653b0c1ac9e42f498cf09ae6133bcbb'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:02.046342 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1104/2206] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:38:01 2022] DELETE /identity/v3/users/9653b0c1ac9e42f498cf09ae6133bcbb => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:38:02.104315 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:DELETE=7,keystone:SELECT=93 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:02.418281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:02.629080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:02.846341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48b2e4-cfdf-41bb-a66c-ee93d4509fd3 service placement] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:02.859647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48b2e4-cfdf-41bb-a66c-ee93d4509fd3 service placement] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:02.959634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e2c22-ee43-4e72-ba17-9d574b1ce191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:02.960406 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e2c22-ee43-4e72-ba17-9d574b1ce191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:02.962213 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e2c22-ee43-4e72-ba17-9d574b1ce191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:02.962660 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e2c22-ee43-4e72-ba17-9d574b1ce191 admin admin] PATH_INFO: `/v3/projects/ca0bc6b0f03945cea752e08c19516031` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:02.964358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e2c22-ee43-4e72-ba17-9d574b1ce191 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca0bc6b0f03945cea752e08c19516031)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:02.966700 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e2c22-ee43-4e72-ba17-9d574b1ce191 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:03.082957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:03.091781 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4a1e2c22-ee43-4e72-ba17-9d574b1ce191 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca0bc6b0f03945cea752e08c19516031', 'request_id': 'req-4a1e2c22-ee43-4e72-ba17-9d574b1ce191'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:03.101488 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4a1e2c22-ee43-4e72-ba17-9d574b1ce191 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca0bc6b0f03945cea752e08c19516031', 'request_id': 'req-4a1e2c22-ee43-4e72-ba17-9d574b1ce191'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:03.112100 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4a1e2c22-ee43-4e72-ba17-9d574b1ce191 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca0bc6b0f03945cea752e08c19516031', 'request_id': 'req-4a1e2c22-ee43-4e72-ba17-9d574b1ce191'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:03.205638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4a1e2c22-ee43-4e72-ba17-9d574b1ce191 admin admin] The token cache is being invalidate because project ca0bc6b0f03945cea752e08c19516031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:38:03.211162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4a1e2c22-ee43-4e72-ba17-9d574b1ce191 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:03.211162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4a1e2c22-ee43-4e72-ba17-9d574b1ce191 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:03.211162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1103/2207] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:38:02 2022] DELETE /identity/v3/projects/ca0bc6b0f03945cea752e08c19516031 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:38:03.213234 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e906c-8554-48e2-ae6d-3b832d9a8d3a admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:03.214291 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-0d6e906c-8554-48e2-ae6d-3b832d9a8d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:03.234593 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e906c-8554-48e2-ae6d-3b832d9a8d3a admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:03.356195 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef74eed-bffe-495a-aa10-897ac4abf6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:03.356860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef74eed-bffe-495a-aa10-897ac4abf6d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:03.357240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef74eed-bffe-495a-aa10-897ac4abf6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:03.357522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef74eed-bffe-495a-aa10-897ac4abf6d6 admin admin] PATH_INFO: `/v3/users/37554a0aebaf4dd486aeeb4ce830d48a` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:03.358859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef74eed-bffe-495a-aa10-897ac4abf6d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37554a0aebaf4dd486aeeb4ce830d48a)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:03.360867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef74eed-bffe-495a-aa10-897ac4abf6d6 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:03.540544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-3ef74eed-bffe-495a-aa10-897ac4abf6d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37554a0aebaf4dd486aeeb4ce830d48a'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:03.634237 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-3ef74eed-bffe-495a-aa10-897ac4abf6d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37554a0aebaf4dd486aeeb4ce830d48a'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:03.635358 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-3ef74eed-bffe-495a-aa10-897ac4abf6d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37554a0aebaf4dd486aeeb4ce830d48a'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:03.637262 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-3ef74eed-bffe-495a-aa10-897ac4abf6d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37554a0aebaf4dd486aeeb4ce830d48a'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:03.646733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-3ef74eed-bffe-495a-aa10-897ac4abf6d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37554a0aebaf4dd486aeeb4ce830d48a'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:03.668887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1105/2208] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:38:03 2022] DELETE /identity/v3/users/37554a0aebaf4dd486aeeb4ce830d48a => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:38:04.034464 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:04.125762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e2c22-ee43-4e72-ba17-9d574b1ce191 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:04.127291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4a1e2c22-ee43-4e72-ba17-9d574b1ce191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:04.140553 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e2c22-ee43-4e72-ba17-9d574b1ce191 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:04.193762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448793d4-30ba-4892-9cf2-5e669f8bee4e None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:04.194601 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448793d4-30ba-4892-9cf2-5e669f8bee4e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:04.194790 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448793d4-30ba-4892-9cf2-5e669f8bee4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:04.198891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:04.223472 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621716f-e30b-4b55-ab2b-fcfcf898cd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:04.224054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621716f-e30b-4b55-ab2b-fcfcf898cd7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:04.224331 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621716f-e30b-4b55-ab2b-fcfcf898cd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:04.224601 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621716f-e30b-4b55-ab2b-fcfcf898cd7c admin admin] PATH_INFO: `/v3/projects/454cc1021dbe4735870f4146232b5fb6` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:04.225821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621716f-e30b-4b55-ab2b-fcfcf898cd7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=454cc1021dbe4735870f4146232b5fb6)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:04.227282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-448793d4-30ba-4892-9cf2-5e669f8bee4e None None] MFA Rules not processed for user `0b3fcdf3a12147f591c0ea67b57c8c7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:38:04.227731 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621716f-e30b-4b55-ab2b-fcfcf898cd7c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:04.228500 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-448793d4-30ba-4892-9cf2-5e669f8bee4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:04.304713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1106/2209] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:38:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:38:04.306017 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448793d4-30ba-4892-9cf2-5e669f8bee4e None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:04.307338 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-448793d4-30ba-4892-9cf2-5e669f8bee4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:04.323792 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448793d4-30ba-4892-9cf2-5e669f8bee4e None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:04.343317 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9621716f-e30b-4b55-ab2b-fcfcf898cd7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '454cc1021dbe4735870f4146232b5fb6', 'request_id': 'req-9621716f-e30b-4b55-ab2b-fcfcf898cd7c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:04.357108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9621716f-e30b-4b55-ab2b-fcfcf898cd7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '454cc1021dbe4735870f4146232b5fb6', 'request_id': 'req-9621716f-e30b-4b55-ab2b-fcfcf898cd7c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:04.366541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9621716f-e30b-4b55-ab2b-fcfcf898cd7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '454cc1021dbe4735870f4146232b5fb6', 'request_id': 'req-9621716f-e30b-4b55-ab2b-fcfcf898cd7c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:04.395498 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f84dc-9a71-44ee-823b-aa0c6d282bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:04.396418 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f84dc-9a71-44ee-823b-aa0c6d282bca admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:04.396892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f84dc-9a71-44ee-823b-aa0c6d282bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:04.397340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f84dc-9a71-44ee-823b-aa0c6d282bca admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:04.398188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f84dc-9a71-44ee-823b-aa0c6d282bca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:04.400307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f84dc-9a71-44ee-823b-aa0c6d282bca admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:04.438803 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9621716f-e30b-4b55-ab2b-fcfcf898cd7c admin admin] The token cache is being invalidate because project 454cc1021dbe4735870f4146232b5fb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:38:04.439342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9621716f-e30b-4b55-ab2b-fcfcf898cd7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:04.440295 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9621716f-e30b-4b55-ab2b-fcfcf898cd7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:38:04.445756 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1104/2210] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:38:04 2022] DELETE /identity/v3/projects/454cc1021dbe4735870f4146232b5fb6 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:38:04.447824 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65694391-ca29-41e6-88db-598c50b9edcf None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:04.448074 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65694391-ca29-41e6-88db-598c50b9edcf None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:04.448353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65694391-ca29-41e6-88db-598c50b9edcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:04.471086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-65694391-ca29-41e6-88db-598c50b9edcf None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:38:04.472466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-65694391-ca29-41e6-88db-598c50b9edcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:04.483729 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1105/2211] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:38:04.493004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a855e1-7afb-4374-9930-17ea08799504 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:04.493745 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a855e1-7afb-4374-9930-17ea08799504 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:04.493981 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a855e1-7afb-4374-9930-17ea08799504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:04.503160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1107/2212] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:38:04 2022] POST /identity/v3/projects => generated 383 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:38:04.508712 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f84dc-9a71-44ee-823b-aa0c6d282bca admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:04.511968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-409f84dc-9a71-44ee-823b-aa0c6d282bca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:04.527355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f5a855e1-7afb-4374-9930-17ea08799504 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:38:04.529543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f5a855e1-7afb-4374-9930-17ea08799504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:04.626073 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f84dc-9a71-44ee-823b-aa0c6d282bca admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:04.632867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f08caa-24bb-4ce5-807e-59f2a527f5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:04.634036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f08caa-24bb-4ce5-807e-59f2a527f5b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:04.634413 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f08caa-24bb-4ce5-807e-59f2a527f5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:04.634954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f08caa-24bb-4ce5-807e-59f2a527f5b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:04.635528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f08caa-24bb-4ce5-807e-59f2a527f5b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:04.637654 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f08caa-24bb-4ce5-807e-59f2a527f5b0 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:04.650139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1106/2213] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:38:04.654368 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a855e1-7afb-4374-9930-17ea08799504 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:04.667055 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a855e1-7afb-4374-9930-17ea08799504 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:04.672022 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc90f0c-68a2-4d5e-90a5-7aeb5f33221e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:04.672628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc90f0c-68a2-4d5e-90a5-7aeb5f33221e admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:04.672904 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc90f0c-68a2-4d5e-90a5-7aeb5f33221e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:04.673234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc90f0c-68a2-4d5e-90a5-7aeb5f33221e admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:04.673884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc90f0c-68a2-4d5e-90a5-7aeb5f33221e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:38:04.674185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc90f0c-68a2-4d5e-90a5-7aeb5f33221e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:04.675831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc90f0c-68a2-4d5e-90a5-7aeb5f33221e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:04.692713 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1107/2214] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:38:04.697500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc90f0c-68a2-4d5e-90a5-7aeb5f33221e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:04.710934 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc90f0c-68a2-4d5e-90a5-7aeb5f33221e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:04.718412 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0e5e1-46c3-4b50-b55b-7dd0339a413e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:04.720228 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0e5e1-46c3-4b50-b55b-7dd0339a413e admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:04.720889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0e5e1-46c3-4b50-b55b-7dd0339a413e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:04.721664 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0e5e1-46c3-4b50-b55b-7dd0339a413e admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:04.723026 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b0e5e1-46c3-4b50-b55b-7dd0339a413e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:04.727939 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b0e5e1-46c3-4b50-b55b-7dd0339a413e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:04.771435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1108/2215] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:38:04 2022] POST /identity/v3/users => generated 298 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:38:04.776403 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f08caa-24bb-4ce5-807e-59f2a527f5b0 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:04.799734 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f08caa-24bb-4ce5-807e-59f2a527f5b0 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:04.807362 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526fe00-f478-4a5b-ae6d-710018e757a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:04.808082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526fe00-f478-4a5b-ae6d-710018e757a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:04.808468 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526fe00-f478-4a5b-ae6d-710018e757a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:04.809031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526fe00-f478-4a5b-ae6d-710018e757a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:04.809913 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526fe00-f478-4a5b-ae6d-710018e757a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:04.811562 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526fe00-f478-4a5b-ae6d-710018e757a0 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:04.827961 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1108/2216] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:38:04 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:38:04.833018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0e5e1-46c3-4b50-b55b-7dd0339a413e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:04.843258 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1109/2217] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:38:04.846944 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526fe00-f478-4a5b-ae6d-710018e757a0 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:04.866123 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526fe00-f478-4a5b-ae6d-710018e757a0 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:04.952176 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0e5e1-46c3-4b50-b55b-7dd0339a413e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:04.962464 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e5e15-4aab-4c82-a73e-2abdf605b432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:04.963213 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e5e15-4aab-4c82-a73e-2abdf605b432 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:04.963706 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e5e15-4aab-4c82-a73e-2abdf605b432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:04.963913 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e5e15-4aab-4c82-a73e-2abdf605b432 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:04.967015 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e5e15-4aab-4c82-a73e-2abdf605b432 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:04.969260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937777c5-de74-4d2c-a9a6-85ecfe37cf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:04.970188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937777c5-de74-4d2c-a9a6-85ecfe37cf88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:04.970400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937777c5-de74-4d2c-a9a6-85ecfe37cf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:04.970684 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937777c5-de74-4d2c-a9a6-85ecfe37cf88 admin admin] PATH_INFO: `/v3/projects/1ced0349fde44cbda660d1dbddb61162/users/75a2146ad007405089cfcd83be256ebd/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:04.970799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e5e15-4aab-4c82-a73e-2abdf605b432 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:04.989636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937777c5-de74-4d2c-a9a6-85ecfe37cf88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ced0349fde44cbda660d1dbddb61162, user_id=75a2146ad007405089cfcd83be256ebd, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:04.994130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937777c5-de74-4d2c-a9a6-85ecfe37cf88 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:05.086143 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1109/2218] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:38:04 2022] POST /identity/v3/users => generated 287 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:38:05.093632 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e5e15-4aab-4c82-a73e-2abdf605b432 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:05.098707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1110/2219] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:38:04 2022] PUT /identity/v3/projects/1ced0349fde44cbda660d1dbddb61162/users/75a2146ad007405089cfcd83be256ebd/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:38:05.103333 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937777c5-de74-4d2c-a9a6-85ecfe37cf88 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:05.119106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e5e15-4aab-4c82-a73e-2abdf605b432 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:05.197830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:05.266759 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d4cde-0808-43e2-b79a-4cd53ae95393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:05.267335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d4cde-0808-43e2-b79a-4cd53ae95393 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:05.267570 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d4cde-0808-43e2-b79a-4cd53ae95393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:05.268361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d4cde-0808-43e2-b79a-4cd53ae95393 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:05.269034 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d4cde-0808-43e2-b79a-4cd53ae95393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:05.270689 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d4cde-0808-43e2-b79a-4cd53ae95393 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:05.278273 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937777c5-de74-4d2c-a9a6-85ecfe37cf88 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:05.289702 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1110/2220] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:38:05.293798 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d4cde-0808-43e2-b79a-4cd53ae95393 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:05.314986 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f44ab-f2ec-4dc0-a9d2-c8fcbd8c6ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:05.315697 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f44ab-f2ec-4dc0-a9d2-c8fcbd8c6ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:05.315964 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f44ab-f2ec-4dc0-a9d2-c8fcbd8c6ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:05.316237 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f44ab-f2ec-4dc0-a9d2-c8fcbd8c6ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:05.316894 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f44ab-f2ec-4dc0-a9d2-c8fcbd8c6ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:05.318595 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f44ab-f2ec-4dc0-a9d2-c8fcbd8c6ba2 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:05.324834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d4cde-0808-43e2-b79a-4cd53ae95393 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:05.336487 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1111/2221] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:38:05.340499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f44ab-f2ec-4dc0-a9d2-c8fcbd8c6ba2 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:05.341673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214a415-56b6-431b-b3ae-a45b9db2cb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:05.342228 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214a415-56b6-431b-b3ae-a45b9db2cb35 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:05.342560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214a415-56b6-431b-b3ae-a45b9db2cb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:05.342936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214a415-56b6-431b-b3ae-a45b9db2cb35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:05.343502 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214a415-56b6-431b-b3ae-a45b9db2cb35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:05.345136 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214a415-56b6-431b-b3ae-a45b9db2cb35 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:05.361525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1111/2222] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:38:05.363354 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214a415-56b6-431b-b3ae-a45b9db2cb35 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:05.368717 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f44ab-f2ec-4dc0-a9d2-c8fcbd8c6ba2 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:05.374669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214a415-56b6-431b-b3ae-a45b9db2cb35 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:05.380492 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdf13b-f018-40f7-905f-cbd3aafe91de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:05.380866 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdf13b-f018-40f7-905f-cbd3aafe91de admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:05.381712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdf13b-f018-40f7-905f-cbd3aafe91de admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:05.382012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdf13b-f018-40f7-905f-cbd3aafe91de admin admin] PATH_INFO: `/v3/projects/993ad6a2ac01432a8d3b64fe272bd874/users/0bc8bbdc05484c239d0dc820b95ac664/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:05.383312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c25c9-62b1-4831-bad5-5b32c14d184c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:05.384643 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c25c9-62b1-4831-bad5-5b32c14d184c admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:05.386727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c25c9-62b1-4831-bad5-5b32c14d184c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:05.387844 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c25c9-62b1-4831-bad5-5b32c14d184c admin admin] PATH_INFO: `/v3/projects/1ced0349fde44cbda660d1dbddb61162/users/75a2146ad007405089cfcd83be256ebd/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:05.388177 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:05.391893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c25c9-62b1-4831-bad5-5b32c14d184c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ced0349fde44cbda660d1dbddb61162, user_id=75a2146ad007405089cfcd83be256ebd, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:05.396088 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c25c9-62b1-4831-bad5-5b32c14d184c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:05.403688 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bdf13b-f018-40f7-905f-cbd3aafe91de admin admin] RBAC: Authorizing `identity:create_grant(project_id=993ad6a2ac01432a8d3b64fe272bd874, user_id=0bc8bbdc05484c239d0dc820b95ac664, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:05.406968 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bdf13b-f018-40f7-905f-cbd3aafe91de admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:05.418034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1112/2223] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:38:05 2022] PUT /identity/v3/projects/1ced0349fde44cbda660d1dbddb61162/users/75a2146ad007405089cfcd83be256ebd/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:38:05.483768 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1112/2224] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:38:05 2022] PUT /identity/v3/projects/993ad6a2ac01432a8d3b64fe272bd874/users/0bc8bbdc05484c239d0dc820b95ac664/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:38:05.501193 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29d7a0-b63d-4a0d-bc7e-648dd4d4d8fd None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:05.501367 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29d7a0-b63d-4a0d-bc7e-648dd4d4d8fd None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:05.501573 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29d7a0-b63d-4a0d-bc7e-648dd4d4d8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:05.557258 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-7a29d7a0-b63d-4a0d-bc7e-648dd4d4d8fd None None] MFA Rules not processed for user `0bc8bbdc05484c239d0dc820b95ac664`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:38:05.558948 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-7a29d7a0-b63d-4a0d-bc7e-648dd4d4d8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:05.655763 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1113/2225] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:38:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:38:05.668440 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdf13b-f018-40f7-905f-cbd3aafe91de admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:05.670709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d6bdf13b-f018-40f7-905f-cbd3aafe91de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:05.750162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdf13b-f018-40f7-905f-cbd3aafe91de admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:05.754556 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1a635-3ed8-441c-8d4b-5e364e26c10f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:05.755034 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1a635-3ed8-441c-8d4b-5e364e26c10f service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:05.755316 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1a635-3ed8-441c-8d4b-5e364e26c10f service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:05.755634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1a635-3ed8-441c-8d4b-5e364e26c10f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:05.765915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1a635-3ed8-441c-8d4b-5e364e26c10f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:05.767510 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1a635-3ed8-441c-8d4b-5e364e26c10f service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:05.778674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1113/2226] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:38:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:38:07.835964 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b67588-435a-4090-8ff9-81c768211ed3 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:07.836931 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b67588-435a-4090-8ff9-81c768211ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:07.837413 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b67588-435a-4090-8ff9-81c768211ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:07.868947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-28b67588-435a-4090-8ff9-81c768211ed3 None None] MFA Rules not processed for user `0bc8bbdc05484c239d0dc820b95ac664`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:38:07.870528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-28b67588-435a-4090-8ff9-81c768211ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:07.881937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1114/2227] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:38:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:38:08.165887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:08.515209 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:10.908452 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:12.187764 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:12.767247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:12.835717 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f51a3a-aeef-4218-be70-074ce7104eb0 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:12.839193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f51a3a-aeef-4218-be70-074ce7104eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:12.839639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f51a3a-aeef-4218-be70-074ce7104eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:12.924136 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-98f51a3a-aeef-4218-be70-074ce7104eb0 None None] MFA Rules not processed for user `75a2146ad007405089cfcd83be256ebd`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:38:12.925590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-98f51a3a-aeef-4218-be70-074ce7104eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:13.075015 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1114/2228] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:38:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:38:13.087941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b67588-435a-4090-8ff9-81c768211ed3 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:13.089556 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-28b67588-435a-4090-8ff9-81c768211ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:13.163796 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b67588-435a-4090-8ff9-81c768211ed3 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:13.172635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb508bc-6a9c-4a7f-bd8a-a75b7220d936 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a40b1ed93a6b4e5fb0e8f907627379a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:13.173405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb508bc-6a9c-4a7f-bd8a-a75b7220d936 service glance] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:13.173870 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb508bc-6a9c-4a7f-bd8a-a75b7220d936 service glance] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:13.174158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb508bc-6a9c-4a7f-bd8a-a75b7220d936 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:13.175958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-3cb508bc-6a9c-4a7f-bd8a-a75b7220d936 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:13.246086 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb508bc-6a9c-4a7f-bd8a-a75b7220d936 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:13.247676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb508bc-6a9c-4a7f-bd8a-a75b7220d936 service glance] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:13.272147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1115/2229] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:38:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:38:13.627559 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ca283-8aec-496d-825f-632357474673 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:13.632789 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ca283-8aec-496d-825f-632357474673 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:13.632789 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ca283-8aec-496d-825f-632357474673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:13.649713 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8e6ca283-8aec-496d-825f-632357474673 None None] MFA Rules not processed for user `4ae237601b4a4aab9e969fedb5ee37d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:38:13.650902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8e6ca283-8aec-496d-825f-632357474673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:13.703109 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1115/2230] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 19 10:38:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:38:13.746605 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb508bc-6a9c-4a7f-bd8a-a75b7220d936 service glance] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:13.748011 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-3cb508bc-6a9c-4a7f-bd8a-a75b7220d936 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:13.831444 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb508bc-6a9c-4a7f-bd8a-a75b7220d936 service glance] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:13.838292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1f42d-ed06-4efb-a018-b71e860003c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17a8bc6c506c45be897bda23eaab485a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:13.839075 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1f42d-ed06-4efb-a018-b71e860003c8 service swift] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:13.839453 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1f42d-ed06-4efb-a018-b71e860003c8 service swift] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:13.839752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1f42d-ed06-4efb-a018-b71e860003c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:13.852785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1f42d-ed06-4efb-a018-b71e860003c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:13.855005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1f42d-ed06-4efb-a018-b71e860003c8 service swift] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:13.864304 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1116/2231] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 19 10:38:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 10:38:13.979133 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:14.008866 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:16.094386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:16.345361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:17.089087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:18.192290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18c951-368f-4e8d-b496-e93ee307f386 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:18.193694 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18c951-368f-4e8d-b496-e93ee307f386 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:18.193694 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18c951-368f-4e8d-b496-e93ee307f386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:18.228740 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6a18c951-368f-4e8d-b496-e93ee307f386 None None] MFA Rules not processed for user `29ae2fba5ddb4b30a7a8a3ecf10d28bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:38:18.231261 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6a18c951-368f-4e8d-b496-e93ee307f386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:18.250844 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:18.325288 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1116/2232] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:38:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:38:18.960660 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:19.771490 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:20.847772 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:22.081882 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:22.328496 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:24.356996 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:24.644386 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:25.159905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d58ed-406a-4cb5-a343-2bfd84ab59b7 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:25.160301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d58ed-406a-4cb5-a343-2bfd84ab59b7 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:25.160696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d58ed-406a-4cb5-a343-2bfd84ab59b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:25.184040 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-716d58ed-406a-4cb5-a343-2bfd84ab59b7 None None] MFA Rules not processed for user `8a059ba74d744a12a4694513f8ca5789`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:38:25.185395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-716d58ed-406a-4cb5-a343-2bfd84ab59b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:25.274748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:25.283707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1117/2233] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:38:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:38:25.753522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:26.090501 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:26.164904 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:26.726325 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:26.853603 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:27.385525 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:28.716741 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:29.770832 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:30.003906 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed6c1c-3d10-4c78-858b-ee9b579e3d81 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:30.004999 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed6c1c-3d10-4c78-858b-ee9b579e3d81 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:30.004999 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed6c1c-3d10-4c78-858b-ee9b579e3d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:30.028274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e5ed6c1c-3d10-4c78-858b-ee9b579e3d81 None None] MFA Rules not processed for user `0b3fcdf3a12147f591c0ea67b57c8c7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:38:30.030913 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e5ed6c1c-3d10-4c78-858b-ee9b579e3d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:30.038487 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1117/2234] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:38:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:38:30.350453 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:30.522726 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:30.616746 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:30.964250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:36.686774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b33451-09a4-47ee-8832-5c56276e3c44 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:36.686774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b33451-09a4-47ee-8832-5c56276e3c44 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:36.686774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b33451-09a4-47ee-8832-5c56276e3c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:36.699753 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-d6b33451-09a4-47ee-8832-5c56276e3c44 None None] MFA Rules not processed for user `4ae237601b4a4aab9e969fedb5ee37d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:38:36.700963 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d6b33451-09a4-47ee-8832-5c56276e3c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:36.709747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1118/2235] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 19 10:38:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:38:36.721686 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed6c1c-3d10-4c78-858b-ee9b579e3d81 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:36.734162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed6c1c-3d10-4c78-858b-ee9b579e3d81 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:36.740498 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7daff79-7482-45a9-a5d8-66739ce8ca15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17a8bc6c506c45be897bda23eaab485a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:36.741170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daff79-7482-45a9-a5d8-66739ce8ca15 service swift] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:36.744136 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daff79-7482-45a9-a5d8-66739ce8ca15 service swift] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:36.744136 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daff79-7482-45a9-a5d8-66739ce8ca15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:36.751011 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7daff79-7482-45a9-a5d8-66739ce8ca15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:36.752576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7daff79-7482-45a9-a5d8-66739ce8ca15 service swift] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:36.762448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1118/2236] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 19 10:38:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 10:38:40.618307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe0652-fd95-4cd1-966b-d1c25b668fdd None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:40.621557 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe0652-fd95-4cd1-966b-d1c25b668fdd None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:40.621557 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe0652-fd95-4cd1-966b-d1c25b668fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:40.671378 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-afbe0652-fd95-4cd1-966b-d1c25b668fdd None None] MFA Rules not processed for user `4ae237601b4a4aab9e969fedb5ee37d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:38:40.671378 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-afbe0652-fd95-4cd1-966b-d1c25b668fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:40.689586 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1119/2237] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 19 10:38:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:38:40.708371 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7daff79-7482-45a9-a5d8-66739ce8ca15 service swift] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:40.719183 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7daff79-7482-45a9-a5d8-66739ce8ca15 service swift] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:40.736441 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e46062-2c1c-4264-9e2e-948b359298d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17a8bc6c506c45be897bda23eaab485a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:40.737423 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e46062-2c1c-4264-9e2e-948b359298d2 service swift] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:40.739463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e46062-2c1c-4264-9e2e-948b359298d2 service swift] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:40.739922 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e46062-2c1c-4264-9e2e-948b359298d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:40.768882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e46062-2c1c-4264-9e2e-948b359298d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:40.771079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e46062-2c1c-4264-9e2e-948b359298d2 service swift] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:40.786732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1119/2238] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 19 10:38:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 10:38:50.639347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=52 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:50.781830 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=83 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:38:53.543607 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c90cec-1781-4de8-9c04-424bda7d1f8a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:53.543607 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c90cec-1781-4de8-9c04-424bda7d1f8a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:53.543607 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c90cec-1781-4de8-9c04-424bda7d1f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:53.577478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-35c90cec-1781-4de8-9c04-424bda7d1f8a None None] MFA Rules not processed for user `4ae237601b4a4aab9e969fedb5ee37d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:38:53.584510 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-35c90cec-1781-4de8-9c04-424bda7d1f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:53.611847 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1120/2239] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 19 10:38:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:38:53.648523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e46062-2c1c-4264-9e2e-948b359298d2 service swift] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:53.661912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e46062-2c1c-4264-9e2e-948b359298d2 service swift] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:53.667424 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36690d-b1ef-4576-9bc5-143653bd2793 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17a8bc6c506c45be897bda23eaab485a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:53.668152 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36690d-b1ef-4576-9bc5-143653bd2793 service swift] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:53.668493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36690d-b1ef-4576-9bc5-143653bd2793 service swift] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:53.668845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36690d-b1ef-4576-9bc5-143653bd2793 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:53.679024 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36690d-b1ef-4576-9bc5-143653bd2793 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:53.680740 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36690d-b1ef-4576-9bc5-143653bd2793 service swift] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:53.690919 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1120/2240] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 19 10:38:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 10:38:55.578667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55eb80b-d974-423b-866a-90ae5dc94834 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:55.579222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55eb80b-d974-423b-866a-90ae5dc94834 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:55.579222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55eb80b-d974-423b-866a-90ae5dc94834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:55.598990 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-e55eb80b-d974-423b-866a-90ae5dc94834 None None] MFA Rules not processed for user `4ae237601b4a4aab9e969fedb5ee37d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:38:55.600188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e55eb80b-d974-423b-866a-90ae5dc94834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:55.607231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1121/2241] 10.222.0.42 () {58 vars in 967 bytes} [Mon Sep 19 10:38:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:38:55.620184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36690d-b1ef-4576-9bc5-143653bd2793 service swift] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:55.631166 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36690d-b1ef-4576-9bc5-143653bd2793 service swift] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:55.636150 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daacf9a-e81c-4a8f-86a2-4f6dd55efa0e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17a8bc6c506c45be897bda23eaab485a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:55.636735 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daacf9a-e81c-4a8f-86a2-4f6dd55efa0e service swift] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:55.637013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daacf9a-e81c-4a8f-86a2-4f6dd55efa0e service swift] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:55.637313 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daacf9a-e81c-4a8f-86a2-4f6dd55efa0e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:55.646031 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daacf9a-e81c-4a8f-86a2-4f6dd55efa0e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:55.647596 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daacf9a-e81c-4a8f-86a2-4f6dd55efa0e service swift] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:55.655494 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1121/2242] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 19 10:38:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 10:38:56.350745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51253779-2f52-4655-a5c1-dbbbff78e67a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:56.351035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51253779-2f52-4655-a5c1-dbbbff78e67a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:56.351314 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51253779-2f52-4655-a5c1-dbbbff78e67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:56.372045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-51253779-2f52-4655-a5c1-dbbbff78e67a None None] MFA Rules not processed for user `4ae237601b4a4aab9e969fedb5ee37d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:38:56.380899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-51253779-2f52-4655-a5c1-dbbbff78e67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:56.382469 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1122/2243] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 19 10:38:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:38:56.398087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daacf9a-e81c-4a8f-86a2-4f6dd55efa0e service swift] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:56.410666 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daacf9a-e81c-4a8f-86a2-4f6dd55efa0e service swift] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:56.416101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f916de-a011-4a86-8127-611b7065c16c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17a8bc6c506c45be897bda23eaab485a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:56.416715 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f916de-a011-4a86-8127-611b7065c16c service swift] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:56.416980 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f916de-a011-4a86-8127-611b7065c16c service swift] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:56.417296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f916de-a011-4a86-8127-611b7065c16c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:56.426384 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f916de-a011-4a86-8127-611b7065c16c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:56.428018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f916de-a011-4a86-8127-611b7065c16c service swift] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:56.436515 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1122/2244] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 19 10:38:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 10:38:57.113019 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c53e6-df23-44f1-86db-1053cfb873fc None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:57.113526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c53e6-df23-44f1-86db-1053cfb873fc None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:57.113729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c53e6-df23-44f1-86db-1053cfb873fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:57.133863 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-5a8c53e6-df23-44f1-86db-1053cfb873fc None None] MFA Rules not processed for user `4ae237601b4a4aab9e969fedb5ee37d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:38:57.135091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5a8c53e6-df23-44f1-86db-1053cfb873fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:38:57.142794 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1123/2245] 10.222.0.42 () {58 vars in 967 bytes} [Mon Sep 19 10:38:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:38:57.156175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f916de-a011-4a86-8127-611b7065c16c service swift] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:38:57.167199 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f916de-a011-4a86-8127-611b7065c16c service swift] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:38:57.172802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282782b-a2cd-4e2b-a00d-5495479ad22c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17a8bc6c506c45be897bda23eaab485a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:38:57.173797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282782b-a2cd-4e2b-a00d-5495479ad22c service swift] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:38:57.174313 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282782b-a2cd-4e2b-a00d-5495479ad22c service swift] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:38:57.174801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282782b-a2cd-4e2b-a00d-5495479ad22c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:38:57.184550 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282782b-a2cd-4e2b-a00d-5495479ad22c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:38:57.186215 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282782b-a2cd-4e2b-a00d-5495479ad22c service swift] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:38:57.195473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1123/2246] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 19 10:38:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 10:39:00.054861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c53e6-df23-44f1-86db-1053cfb873fc None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:00.149720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c53e6-df23-44f1-86db-1053cfb873fc None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:00.156473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39948544-5fb7-4744-ab40-f85611b165dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:00.157918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39948544-5fb7-4744-ab40-f85611b165dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:00.158404 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39948544-5fb7-4744-ab40-f85611b165dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:00.159061 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39948544-5fb7-4744-ab40-f85611b165dd admin admin] PATH_INFO: `/v3/users/0bc8bbdc05484c239d0dc820b95ac664` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:00.160526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39948544-5fb7-4744-ab40-f85611b165dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bc8bbdc05484c239d0dc820b95ac664)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:00.162862 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39948544-5fb7-4744-ab40-f85611b165dd admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:00.372055 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-39948544-5fb7-4744-ab40-f85611b165dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bc8bbdc05484c239d0dc820b95ac664'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:00.476843 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-39948544-5fb7-4744-ab40-f85611b165dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bc8bbdc05484c239d0dc820b95ac664'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:00.477440 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-39948544-5fb7-4744-ab40-f85611b165dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bc8bbdc05484c239d0dc820b95ac664'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:00.478195 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-39948544-5fb7-4744-ab40-f85611b165dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bc8bbdc05484c239d0dc820b95ac664'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:00.485915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-39948544-5fb7-4744-ab40-f85611b165dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bc8bbdc05484c239d0dc820b95ac664'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:00.503115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1124/2247] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:39:00 2022] DELETE /identity/v3/users/0bc8bbdc05484c239d0dc820b95ac664 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:39:00.507707 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5be3e9-26af-475b-b633-55ea209d62cd None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:00.508151 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5be3e9-26af-475b-b633-55ea209d62cd None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:00.508284 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5be3e9-26af-475b-b633-55ea209d62cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:00.527829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-ab5be3e9-26af-475b-b633-55ea209d62cd None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:39:00.529032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ab5be3e9-26af-475b-b633-55ea209d62cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:00.588785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1124/2248] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:39:00.598986 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39948544-5fb7-4744-ab40-f85611b165dd admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:00.600383 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-39948544-5fb7-4744-ab40-f85611b165dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:00.657217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39948544-5fb7-4744-ab40-f85611b165dd admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:00.660777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd774160-5287-438b-97f3-7cdbf56ac92c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:00.661409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd774160-5287-438b-97f3-7cdbf56ac92c service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:00.661690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd774160-5287-438b-97f3-7cdbf56ac92c service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:00.661983 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd774160-5287-438b-97f3-7cdbf56ac92c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:00.668977 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:39:00.671166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd774160-5287-438b-97f3-7cdbf56ac92c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:00.672585 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd774160-5287-438b-97f3-7cdbf56ac92c service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:00.682009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1125/2249] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:39:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:39:00.927066 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5be3e9-26af-475b-b633-55ea209d62cd None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:00.930723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ab5be3e9-26af-475b-b633-55ea209d62cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:00.941024 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5be3e9-26af-475b-b633-55ea209d62cd None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:00.945887 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9073b15-9b6c-46c0-a654-d6a54337756c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:00.946535 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9073b15-9b6c-46c0-a654-d6a54337756c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:00.946816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9073b15-9b6c-46c0-a654-d6a54337756c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:00.947092 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9073b15-9b6c-46c0-a654-d6a54337756c admin admin] PATH_INFO: `/v3/projects/993ad6a2ac01432a8d3b64fe272bd874` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:00.948231 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9073b15-9b6c-46c0-a654-d6a54337756c admin admin] RBAC: Authorizing `identity:delete_project(project_id=993ad6a2ac01432a8d3b64fe272bd874)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:00.950183 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9073b15-9b6c-46c0-a654-d6a54337756c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:01.093664 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f9073b15-9b6c-46c0-a654-d6a54337756c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '993ad6a2ac01432a8d3b64fe272bd874', 'request_id': 'req-f9073b15-9b6c-46c0-a654-d6a54337756c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:01.107849 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f9073b15-9b6c-46c0-a654-d6a54337756c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '993ad6a2ac01432a8d3b64fe272bd874', 'request_id': 'req-f9073b15-9b6c-46c0-a654-d6a54337756c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:01.125405 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f9073b15-9b6c-46c0-a654-d6a54337756c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '993ad6a2ac01432a8d3b64fe272bd874', 'request_id': 'req-f9073b15-9b6c-46c0-a654-d6a54337756c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:01.195777 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f9073b15-9b6c-46c0-a654-d6a54337756c admin admin] The token cache is being invalidate because project 993ad6a2ac01432a8d3b64fe272bd874 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:39:01.208694 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f9073b15-9b6c-46c0-a654-d6a54337756c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:01.208694 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f9073b15-9b6c-46c0-a654-d6a54337756c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:01.208694 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1125/2250] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:39:00 2022] DELETE /identity/v3/projects/993ad6a2ac01432a8d3b64fe272bd874 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:39:06.539292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1319ef9-54c2-4fa1-b530-2158efcec374 None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:06.539784 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1319ef9-54c2-4fa1-b530-2158efcec374 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:06.539898 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1319ef9-54c2-4fa1-b530-2158efcec374 None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:06.541268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1126/2251] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 19 10:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:39:06.547013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a99b9-5529-4e3d-ada8-8e0b980ac91c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:06.547463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a99b9-5529-4e3d-ada8-8e0b980ac91c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:06.547580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a99b9-5529-4e3d-ada8-8e0b980ac91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:06.569500 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-ae3a99b9-5529-4e3d-ada8-8e0b980ac91c None None] MFA Rules not processed for user `b4007794a30247c4a318af62d05e3b92`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:39:06.570715 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ae3a99b9-5529-4e3d-ada8-8e0b980ac91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:06.597200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:39:06.641434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1126/2252] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 19 10:39:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:39:06.652974 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1319ef9-54c2-4fa1-b530-2158efcec374 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:06.654723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d1319ef9-54c2-4fa1-b530-2158efcec374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:06.713090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1319ef9-54c2-4fa1-b530-2158efcec374 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:06.716662 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4cf621-7a4d-43ca-9aec-313fb2495ef5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a40b1ed93a6b4e5fb0e8f907627379a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:06.717300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cf621-7a4d-43ca-9aec-313fb2495ef5 service glance] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:06.717585 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cf621-7a4d-43ca-9aec-313fb2495ef5 service glance] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:06.717859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cf621-7a4d-43ca-9aec-313fb2495ef5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:06.728889 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4cf621-7a4d-43ca-9aec-313fb2495ef5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:06.730730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4cf621-7a4d-43ca-9aec-313fb2495ef5 service glance] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:06.742356 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1127/2253] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:39:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:39:06.961373 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd25215-f258-488f-b638-e04a0f89a6f0 None None] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:06.961657 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd25215-f258-488f-b638-e04a0f89a6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:06.961957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd25215-f258-488f-b638-e04a0f89a6f0 None None] PATH_INFO: `/` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:06.963355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1127/2254] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 19 10:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:39:06.967970 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87a930-10fe-4baf-99e2-aadf47913cde None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:06.968250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87a930-10fe-4baf-99e2-aadf47913cde None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:06.968523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87a930-10fe-4baf-99e2-aadf47913cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:06.988190 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-1b87a930-10fe-4baf-99e2-aadf47913cde None None] MFA Rules not processed for user `a282e74d177544bbbfba0fdb5e52df60`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:39:06.989416 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-1b87a930-10fe-4baf-99e2-aadf47913cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:07.039984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1128/2255] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 19 10:39:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:39:07.052442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd25215-f258-488f-b638-e04a0f89a6f0 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:07.054168 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0fd25215-f258-488f-b638-e04a0f89a6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:07.067372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd25215-f258-488f-b638-e04a0f89a6f0 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:07.070864 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc274545-0d01-45b7-887e-42342c38e773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:07.071450 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274545-0d01-45b7-887e-42342c38e773 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:07.071727 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274545-0d01-45b7-887e-42342c38e773 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:07.072001 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274545-0d01-45b7-887e-42342c38e773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:07.078877 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc274545-0d01-45b7-887e-42342c38e773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:07.080314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc274545-0d01-45b7-887e-42342c38e773 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:07.087907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1128/2256] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:39:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:39:07.172646 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e32dfe-084b-453d-b373-5645b912931f None None] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:07.172956 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e32dfe-084b-453d-b373-5645b912931f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:07.173250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e32dfe-084b-453d-b373-5645b912931f None None] PATH_INFO: `/` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:07.174655 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1129/2257] 10.222.0.42 () {54 vars in 921 bytes} [Mon Sep 19 10:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 10:39:07.180129 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fc69e-02a3-4801-b980-5712d09f936d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:07.180416 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fc69e-02a3-4801-b980-5712d09f936d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:07.180693 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fc69e-02a3-4801-b980-5712d09f936d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:07.200007 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-348fc69e-02a3-4801-b980-5712d09f936d None None] MFA Rules not processed for user `b4007794a30247c4a318af62d05e3b92`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:39:07.201244 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-348fc69e-02a3-4801-b980-5712d09f936d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:07.211033 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1129/2258] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Sep 19 10:39:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:39:07.223780 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e32dfe-084b-453d-b373-5645b912931f None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:07.225558 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d3e32dfe-084b-453d-b373-5645b912931f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:07.239580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e32dfe-084b-453d-b373-5645b912931f None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:07.243836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08bc995-c0cf-4465-bcf3-4509e520b8fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:07.244445 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bc995-c0cf-4465-bcf3-4509e520b8fc service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:07.244725 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bc995-c0cf-4465-bcf3-4509e520b8fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:07.244995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bc995-c0cf-4465-bcf3-4509e520b8fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:07.255547 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08bc995-c0cf-4465-bcf3-4509e520b8fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:07.257005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08bc995-c0cf-4465-bcf3-4509e520b8fc service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:07.269147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1130/2259] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:39:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:39:09.494854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c1030-f08e-494e-bd8e-1a61a1330ebb None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:09.495303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c1030-f08e-494e-bd8e-1a61a1330ebb None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:09.495408 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c1030-f08e-494e-bd8e-1a61a1330ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:09.519777 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-521c1030-f08e-494e-bd8e-1a61a1330ebb None None] MFA Rules not processed for user `4ae237601b4a4aab9e969fedb5ee37d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:39:09.520985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-521c1030-f08e-494e-bd8e-1a61a1330ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:09.580750 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1130/2260] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 19 10:39:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:39:09.596063 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08bc995-c0cf-4465-bcf3-4509e520b8fc service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:09.597575 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e08bc995-c0cf-4465-bcf3-4509e520b8fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:09.682852 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08bc995-c0cf-4465-bcf3-4509e520b8fc service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:09.687973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a49ce-6c1b-43ca-b82f-3f3617a8b212 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17a8bc6c506c45be897bda23eaab485a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:09.688602 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a49ce-6c1b-43ca-b82f-3f3617a8b212 service swift] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:09.688884 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a49ce-6c1b-43ca-b82f-3f3617a8b212 service swift] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:09.689728 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a49ce-6c1b-43ca-b82f-3f3617a8b212 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:09.698940 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a49ce-6c1b-43ca-b82f-3f3617a8b212 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:09.700575 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a49ce-6c1b-43ca-b82f-3f3617a8b212 service swift] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:09.710502 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1131/2261] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 19 10:39:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 10:39:13.795316 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e2550-1bc3-4792-b639-4415e9b9edb9 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:13.795951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e2550-1bc3-4792-b639-4415e9b9edb9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:13.795951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e2550-1bc3-4792-b639-4415e9b9edb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:13.834881 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-e16e2550-1bc3-4792-b639-4415e9b9edb9 None None] MFA Rules not processed for user `4ae237601b4a4aab9e969fedb5ee37d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:39:13.836116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e16e2550-1bc3-4792-b639-4415e9b9edb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:13.849275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1131/2262] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 19 10:39:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:39:13.859857 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a49ce-6c1b-43ca-b82f-3f3617a8b212 service swift] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:13.872276 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a49ce-6c1b-43ca-b82f-3f3617a8b212 service swift] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:13.878481 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a8889-8f7b-4850-95be-beaf5d5c58ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17a8bc6c506c45be897bda23eaab485a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:13.882540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a8889-8f7b-4850-95be-beaf5d5c58ba service swift] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:13.882540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a8889-8f7b-4850-95be-beaf5d5c58ba service swift] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:13.882540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a8889-8f7b-4850-95be-beaf5d5c58ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:13.890528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a8889-8f7b-4850-95be-beaf5d5c58ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:13.892550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a8889-8f7b-4850-95be-beaf5d5c58ba service swift] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:13.909327 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1132/2263] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 19 10:39:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 10:39:23.829723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=54 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:39:23.899703 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=95 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:39:28.267226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c9adb-7b19-43b6-aab8-5a1de8766b5a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:28.267761 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c9adb-7b19-43b6-aab8-5a1de8766b5a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:28.267761 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c9adb-7b19-43b6-aab8-5a1de8766b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:28.287069 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-456c9adb-7b19-43b6-aab8-5a1de8766b5a None None] MFA Rules not processed for user `4ae237601b4a4aab9e969fedb5ee37d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:39:28.288259 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-456c9adb-7b19-43b6-aab8-5a1de8766b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:28.295301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1132/2264] 10.222.0.42 () {58 vars in 967 bytes} [Mon Sep 19 10:39:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:39:28.307571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a8889-8f7b-4850-95be-beaf5d5c58ba service swift] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:28.318963 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a8889-8f7b-4850-95be-beaf5d5c58ba service swift] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:28.324102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e6be1-036e-43d0-be68-8729f8fe53f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17a8bc6c506c45be897bda23eaab485a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:28.324694 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e6be1-036e-43d0-be68-8729f8fe53f0 service swift] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:28.324973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e6be1-036e-43d0-be68-8729f8fe53f0 service swift] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:28.325277 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e6be1-036e-43d0-be68-8729f8fe53f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:28.334075 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e6be1-036e-43d0-be68-8729f8fe53f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:28.335657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e6be1-036e-43d0-be68-8729f8fe53f0 service swift] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:28.344501 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1133/2265] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 19 10:39:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 10:39:31.234353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c9adb-7b19-43b6-aab8-5a1de8766b5a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:31.235736 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-456c9adb-7b19-43b6-aab8-5a1de8766b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:31.300873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c9adb-7b19-43b6-aab8-5a1de8766b5a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:31.305481 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05424c-5be4-44c3-97c2-b604b2966f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:31.306102 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05424c-5be4-44c3-97c2-b604b2966f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:31.306380 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05424c-5be4-44c3-97c2-b604b2966f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:31.306653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05424c-5be4-44c3-97c2-b604b2966f84 admin admin] PATH_INFO: `/v3/users/29ae2fba5ddb4b30a7a8a3ecf10d28bb` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:31.307779 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05424c-5be4-44c3-97c2-b604b2966f84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29ae2fba5ddb4b30a7a8a3ecf10d28bb)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:31.309759 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05424c-5be4-44c3-97c2-b604b2966f84 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:31.529323 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ac05424c-5be4-44c3-97c2-b604b2966f84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29ae2fba5ddb4b30a7a8a3ecf10d28bb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:31.529938 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ac05424c-5be4-44c3-97c2-b604b2966f84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29ae2fba5ddb4b30a7a8a3ecf10d28bb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:31.615312 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ac05424c-5be4-44c3-97c2-b604b2966f84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29ae2fba5ddb4b30a7a8a3ecf10d28bb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:31.615975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ac05424c-5be4-44c3-97c2-b604b2966f84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29ae2fba5ddb4b30a7a8a3ecf10d28bb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:31.628776 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ac05424c-5be4-44c3-97c2-b604b2966f84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29ae2fba5ddb4b30a7a8a3ecf10d28bb'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:31.642929 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1133/2266] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:39:31 2022] DELETE /identity/v3/users/29ae2fba5ddb4b30a7a8a3ecf10d28bb => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:39:31.897259 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e6be1-036e-43d0-be68-8729f8fe53f0 service swift] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:31.910005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e6be1-036e-43d0-be68-8729f8fe53f0 service swift] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:31.967110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca14e0-0e01-4d49-8bec-015cd17f2fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:31.967736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca14e0-0e01-4d49-8bec-015cd17f2fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:31.968018 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca14e0-0e01-4d49-8bec-015cd17f2fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:31.968292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca14e0-0e01-4d49-8bec-015cd17f2fa7 admin admin] PATH_INFO: `/v3/projects/90c8dde9309d4f128f53cd6ece43ea52` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:31.969564 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca14e0-0e01-4d49-8bec-015cd17f2fa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90c8dde9309d4f128f53cd6ece43ea52)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:31.971478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca14e0-0e01-4d49-8bec-015cd17f2fa7 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:32.094541 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-a7ca14e0-0e01-4d49-8bec-015cd17f2fa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90c8dde9309d4f128f53cd6ece43ea52', 'request_id': 'req-a7ca14e0-0e01-4d49-8bec-015cd17f2fa7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:32.102127 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-a7ca14e0-0e01-4d49-8bec-015cd17f2fa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90c8dde9309d4f128f53cd6ece43ea52', 'request_id': 'req-a7ca14e0-0e01-4d49-8bec-015cd17f2fa7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:32.113347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-a7ca14e0-0e01-4d49-8bec-015cd17f2fa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90c8dde9309d4f128f53cd6ece43ea52', 'request_id': 'req-a7ca14e0-0e01-4d49-8bec-015cd17f2fa7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:32.162429 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-a7ca14e0-0e01-4d49-8bec-015cd17f2fa7 admin admin] The token cache is being invalidate because project 90c8dde9309d4f128f53cd6ece43ea52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:39:32.162728 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-a7ca14e0-0e01-4d49-8bec-015cd17f2fa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:32.163702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-a7ca14e0-0e01-4d49-8bec-015cd17f2fa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:39:32.165619 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1134/2267] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:39:31 2022] DELETE /identity/v3/projects/90c8dde9309d4f128f53cd6ece43ea52 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:39:32.170649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcbfc6-f428-491f-81bc-66e6ea9ac970 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:32.172017 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcbfc6-f428-491f-81bc-66e6ea9ac970 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:32.172017 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcbfc6-f428-491f-81bc-66e6ea9ac970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:32.195397 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-72dcbfc6-f428-491f-81bc-66e6ea9ac970 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:39:32.196600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-72dcbfc6-f428-491f-81bc-66e6ea9ac970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:32.256676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1134/2268] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:39:32.264850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57696b-0abd-44f1-a21f-f5fc27e0d8b8 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:32.265173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57696b-0abd-44f1-a21f-f5fc27e0d8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:32.265451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57696b-0abd-44f1-a21f-f5fc27e0d8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:32.289357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-7c57696b-0abd-44f1-a21f-f5fc27e0d8b8 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:39:32.290570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-7c57696b-0abd-44f1-a21f-f5fc27e0d8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:32.299447 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1135/2269] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:39:32.303669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcbfc6-f428-491f-81bc-66e6ea9ac970 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:32.313137 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcbfc6-f428-491f-81bc-66e6ea9ac970 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:32.317841 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332d51f-61ad-4d6d-90e8-1f19bbd2f9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:32.318446 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332d51f-61ad-4d6d-90e8-1f19bbd2f9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:32.318724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332d51f-61ad-4d6d-90e8-1f19bbd2f9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:32.318997 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332d51f-61ad-4d6d-90e8-1f19bbd2f9f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:32.319644 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332d51f-61ad-4d6d-90e8-1f19bbd2f9f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:39:32.319950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332d51f-61ad-4d6d-90e8-1f19bbd2f9f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:32.321573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332d51f-61ad-4d6d-90e8-1f19bbd2f9f9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:32.330685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:39:32.334070 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1135/2270] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:39:32.340684 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57696b-0abd-44f1-a21f-f5fc27e0d8b8 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:32.351562 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57696b-0abd-44f1-a21f-f5fc27e0d8b8 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:32.356486 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87446c7-2f08-49d6-b59c-9277e5fdd552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:32.357148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87446c7-2f08-49d6-b59c-9277e5fdd552 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:32.357429 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87446c7-2f08-49d6-b59c-9277e5fdd552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:32.357702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87446c7-2f08-49d6-b59c-9277e5fdd552 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:32.358355 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87446c7-2f08-49d6-b59c-9277e5fdd552 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:32.360908 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87446c7-2f08-49d6-b59c-9277e5fdd552 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:32.480477 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1136/2271] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:39:32 2022] POST /identity/v3/projects => generated 369 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:39:32.484614 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332d51f-61ad-4d6d-90e8-1f19bbd2f9f9 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:32.495292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332d51f-61ad-4d6d-90e8-1f19bbd2f9f9 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:32.552394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb1ab5-b874-4e73-873f-306593461bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:32.553000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb1ab5-b874-4e73-873f-306593461bfe admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:32.553310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb1ab5-b874-4e73-873f-306593461bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:32.553591 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb1ab5-b874-4e73-873f-306593461bfe admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:32.554278 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb1ab5-b874-4e73-873f-306593461bfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:32.556278 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb1ab5-b874-4e73-873f-306593461bfe admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:32.655300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1136/2272] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:39:32 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:39:32.659896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87446c7-2f08-49d6-b59c-9277e5fdd552 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:32.670127 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87446c7-2f08-49d6-b59c-9277e5fdd552 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:32.674793 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec747b7b-a0af-4508-91a6-eb47066c6a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:32.675386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec747b7b-a0af-4508-91a6-eb47066c6a07 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:32.675660 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec747b7b-a0af-4508-91a6-eb47066c6a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:32.675929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec747b7b-a0af-4508-91a6-eb47066c6a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:32.676562 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec747b7b-a0af-4508-91a6-eb47066c6a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:32.678183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec747b7b-a0af-4508-91a6-eb47066c6a07 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:32.687940 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1137/2273] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:39:32.691819 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb1ab5-b874-4e73-873f-306593461bfe admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:32.701589 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb1ab5-b874-4e73-873f-306593461bfe admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:32.706664 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca58efe-7eca-4578-955c-9794e3543cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:32.707251 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca58efe-7eca-4578-955c-9794e3543cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:32.707532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca58efe-7eca-4578-955c-9794e3543cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:32.707804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca58efe-7eca-4578-955c-9794e3543cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:32.708437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca58efe-7eca-4578-955c-9794e3543cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:32.710032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca58efe-7eca-4578-955c-9794e3543cb8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:32.721022 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1137/2274] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:39:32.724782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec747b7b-a0af-4508-91a6-eb47066c6a07 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:32.734635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec747b7b-a0af-4508-91a6-eb47066c6a07 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:32.739318 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a224cb-ee3a-4271-9a07-02b805b25e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:32.739962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a224cb-ee3a-4271-9a07-02b805b25e44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:32.740238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a224cb-ee3a-4271-9a07-02b805b25e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:32.740507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a224cb-ee3a-4271-9a07-02b805b25e44 admin admin] PATH_INFO: `/v3/projects/987fcb826a2a44dcb8a1919f64e2a726/users/d9d9649bbd494fa48cf9393e75c50a5d/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:32.755497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a224cb-ee3a-4271-9a07-02b805b25e44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=987fcb826a2a44dcb8a1919f64e2a726, user_id=d9d9649bbd494fa48cf9393e75c50a5d, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:32.758180 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a224cb-ee3a-4271-9a07-02b805b25e44 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:32.827627 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1138/2275] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:39:32 2022] PUT /identity/v3/projects/987fcb826a2a44dcb8a1919f64e2a726/users/d9d9649bbd494fa48cf9393e75c50a5d/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:39:32.839101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7da7b-6080-45fc-b071-62e119ac2e37 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:32.839388 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7da7b-6080-45fc-b071-62e119ac2e37 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:32.839666 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7da7b-6080-45fc-b071-62e119ac2e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:32.868237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-95a7da7b-6080-45fc-b071-62e119ac2e37 None None] MFA Rules not processed for user `d9d9649bbd494fa48cf9393e75c50a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:39:32.869489 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-95a7da7b-6080-45fc-b071-62e119ac2e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:32.932104 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1138/2276] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:39:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:39:32.944146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a224cb-ee3a-4271-9a07-02b805b25e44 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:32.945946 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-72a224cb-ee3a-4271-9a07-02b805b25e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:32.960721 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:39:33.002469 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a224cb-ee3a-4271-9a07-02b805b25e44 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:33.006243 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cded33f-bdeb-4dda-88c1-b9dbc710d0dd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a40b1ed93a6b4e5fb0e8f907627379a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:33.006845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cded33f-bdeb-4dda-88c1-b9dbc710d0dd service glance] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:33.007125 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cded33f-bdeb-4dda-88c1-b9dbc710d0dd service glance] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:33.007431 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cded33f-bdeb-4dda-88c1-b9dbc710d0dd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:33.016532 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cded33f-bdeb-4dda-88c1-b9dbc710d0dd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:33.018082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cded33f-bdeb-4dda-88c1-b9dbc710d0dd service glance] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:33.026919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1139/2277] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:39:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:39:36.557689 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7da7b-6080-45fc-b071-62e119ac2e37 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:36.558992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-95a7da7b-6080-45fc-b071-62e119ac2e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:36.619158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7da7b-6080-45fc-b071-62e119ac2e37 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:36.622740 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109aa5c-c314-46ed-9b79-ed73d9eb3c4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:36.623360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109aa5c-c314-46ed-9b79-ed73d9eb3c4a service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:36.623636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109aa5c-c314-46ed-9b79-ed73d9eb3c4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:36.623909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109aa5c-c314-46ed-9b79-ed73d9eb3c4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:36.625931 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7109aa5c-c314-46ed-9b79-ed73d9eb3c4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:36.648706 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:39:36.691823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109aa5c-c314-46ed-9b79-ed73d9eb3c4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:36.693304 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109aa5c-c314-46ed-9b79-ed73d9eb3c4a service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:36.703430 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1139/2278] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:39:36 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:39:43.086378 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=20 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:39:44.311526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2b91f-24aa-4816-966c-258d6b8e0997 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:44.312163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2b91f-24aa-4816-966c-258d6b8e0997 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:44.312163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2b91f-24aa-4816-966c-258d6b8e0997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:44.332427 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-bfd2b91f-24aa-4816-966c-258d6b8e0997 None None] MFA Rules not processed for user `4ae237601b4a4aab9e969fedb5ee37d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:39:44.334188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-bfd2b91f-24aa-4816-966c-258d6b8e0997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:44.397824 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1140/2279] 10.222.0.42 () {58 vars in 967 bytes} [Mon Sep 19 10:39:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:39:44.411673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109aa5c-c314-46ed-9b79-ed73d9eb3c4a service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:44.413934 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7109aa5c-c314-46ed-9b79-ed73d9eb3c4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:44.559517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109aa5c-c314-46ed-9b79-ed73d9eb3c4a service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:44.566762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9f50f-1df3-471f-8b4a-0f106f1119ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17a8bc6c506c45be897bda23eaab485a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:44.569497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9f50f-1df3-471f-8b4a-0f106f1119ea service swift] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:44.569497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9f50f-1df3-471f-8b4a-0f106f1119ea service swift] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:44.569497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9f50f-1df3-471f-8b4a-0f106f1119ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:44.578365 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9f50f-1df3-471f-8b4a-0f106f1119ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:44.580616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9f50f-1df3-471f-8b4a-0f106f1119ea service swift] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:44.590019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1140/2280] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 19 10:39:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 10:39:45.493875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed7a88-7866-4894-af76-1bfbaeccb22a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:45.494396 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed7a88-7866-4894-af76-1bfbaeccb22a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:45.494588 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed7a88-7866-4894-af76-1bfbaeccb22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:45.515578 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-e6ed7a88-7866-4894-af76-1bfbaeccb22a None None] MFA Rules not processed for user `d3235496c38249a388e4afc263eadbc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:39:45.516255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e6ed7a88-7866-4894-af76-1bfbaeccb22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:45.578638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1141/2281] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:39:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:39:47.884110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9f50f-1df3-471f-8b4a-0f106f1119ea service swift] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:39:47.885616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-88c9f50f-1df3-471f-8b4a-0f106f1119ea service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:47.955926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9f50f-1df3-471f-8b4a-0f106f1119ea service swift] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:39:47.961874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597e9ef-b91e-4b06-96f2-b18fc3c8dcec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0ef47a60f54ca1853fd267e308c58b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:39:47.962469 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597e9ef-b91e-4b06-96f2-b18fc3c8dcec service placement] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:39:47.962744 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597e9ef-b91e-4b06-96f2-b18fc3c8dcec service placement] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:39:47.963016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597e9ef-b91e-4b06-96f2-b18fc3c8dcec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:39:47.964698 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7597e9ef-b91e-4b06-96f2-b18fc3c8dcec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:39:47.976184 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597e9ef-b91e-4b06-96f2-b18fc3c8dcec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:39:47.977890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597e9ef-b91e-4b06-96f2-b18fc3c8dcec service placement] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:39:47.987713 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1141/2282] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:39:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:39:55.568692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=50 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:39:57.983558 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=83 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:40:13.713086 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed7a88-7866-4894-af76-1bfbaeccb22a None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:13.778460 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed7a88-7866-4894-af76-1bfbaeccb22a None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:13.783662 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76999c68-ec62-4c56-8c67-eb86092c6052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:13.784400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76999c68-ec62-4c56-8c67-eb86092c6052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:13.784687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76999c68-ec62-4c56-8c67-eb86092c6052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:13.784966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76999c68-ec62-4c56-8c67-eb86092c6052 admin admin] PATH_INFO: `/v3/users/d9d9649bbd494fa48cf9393e75c50a5d` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:13.786354 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76999c68-ec62-4c56-8c67-eb86092c6052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9d9649bbd494fa48cf9393e75c50a5d)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:13.788332 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76999c68-ec62-4c56-8c67-eb86092c6052 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:13.949543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-76999c68-ec62-4c56-8c67-eb86092c6052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9d9649bbd494fa48cf9393e75c50a5d'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:14.025194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-76999c68-ec62-4c56-8c67-eb86092c6052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9d9649bbd494fa48cf9393e75c50a5d'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:14.025978 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-76999c68-ec62-4c56-8c67-eb86092c6052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9d9649bbd494fa48cf9393e75c50a5d'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:14.026872 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-76999c68-ec62-4c56-8c67-eb86092c6052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9d9649bbd494fa48cf9393e75c50a5d'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:14.035157 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-76999c68-ec62-4c56-8c67-eb86092c6052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9d9649bbd494fa48cf9393e75c50a5d'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:14.061743 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1142/2283] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:40:13 2022] DELETE /identity/v3/users/d9d9649bbd494fa48cf9393e75c50a5d => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:40:14.066153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212093da-a423-4497-a4bf-a4c21eeb8f64 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:14.067075 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212093da-a423-4497-a4bf-a4c21eeb8f64 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:14.067356 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212093da-a423-4497-a4bf-a4c21eeb8f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:14.087068 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-212093da-a423-4497-a4bf-a4c21eeb8f64 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:40:14.088301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-212093da-a423-4497-a4bf-a4c21eeb8f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:14.155913 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1142/2284] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:40:14.166818 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76999c68-ec62-4c56-8c67-eb86092c6052 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:14.168458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-76999c68-ec62-4c56-8c67-eb86092c6052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:14.229614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76999c68-ec62-4c56-8c67-eb86092c6052 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:14.233327 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff0b25-62ba-43ac-bdc5-b0748e517f0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:14.233941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff0b25-62ba-43ac-bdc5-b0748e517f0f service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:14.234222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff0b25-62ba-43ac-bdc5-b0748e517f0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:14.234555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff0b25-62ba-43ac-bdc5-b0748e517f0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:14.244959 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff0b25-62ba-43ac-bdc5-b0748e517f0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:14.246444 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff0b25-62ba-43ac-bdc5-b0748e517f0f service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:14.257301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1143/2285] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:40:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:40:14.753233 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212093da-a423-4497-a4bf-a4c21eeb8f64 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:14.756244 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-212093da-a423-4497-a4bf-a4c21eeb8f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:14.790935 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212093da-a423-4497-a4bf-a4c21eeb8f64 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:14.794762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa3e7c-c81d-44c3-9444-709b24dc69e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:14.795404 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa3e7c-c81d-44c3-9444-709b24dc69e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:14.796010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa3e7c-c81d-44c3-9444-709b24dc69e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:14.796354 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa3e7c-c81d-44c3-9444-709b24dc69e7 admin admin] PATH_INFO: `/v3/projects/987fcb826a2a44dcb8a1919f64e2a726` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:14.800048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa3e7c-c81d-44c3-9444-709b24dc69e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=987fcb826a2a44dcb8a1919f64e2a726)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:14.802133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa3e7c-c81d-44c3-9444-709b24dc69e7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:14.985320 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-02aa3e7c-c81d-44c3-9444-709b24dc69e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '987fcb826a2a44dcb8a1919f64e2a726', 'request_id': 'req-02aa3e7c-c81d-44c3-9444-709b24dc69e7'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:14.990925 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-02aa3e7c-c81d-44c3-9444-709b24dc69e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '987fcb826a2a44dcb8a1919f64e2a726', 'request_id': 'req-02aa3e7c-c81d-44c3-9444-709b24dc69e7'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:14.997655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-02aa3e7c-c81d-44c3-9444-709b24dc69e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '987fcb826a2a44dcb8a1919f64e2a726', 'request_id': 'req-02aa3e7c-c81d-44c3-9444-709b24dc69e7'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:15.075672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-02aa3e7c-c81d-44c3-9444-709b24dc69e7 admin admin] The token cache is being invalidate because project 987fcb826a2a44dcb8a1919f64e2a726 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:40:15.076320 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-02aa3e7c-c81d-44c3-9444-709b24dc69e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:15.076795 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-02aa3e7c-c81d-44c3-9444-709b24dc69e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:15.078121 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1143/2286] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:40:14 2022] DELETE /identity/v3/projects/987fcb826a2a44dcb8a1919f64e2a726 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:40:15.083058 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46338db6-4996-4f22-becf-7f0731a19c7d None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:15.084078 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46338db6-4996-4f22-becf-7f0731a19c7d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:15.084225 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46338db6-4996-4f22-becf-7f0731a19c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:15.103691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-46338db6-4996-4f22-becf-7f0731a19c7d None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:40:15.104932 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-46338db6-4996-4f22-becf-7f0731a19c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:15.122852 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:40:15.165409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1144/2287] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:40:15.173359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534b3f1-b07e-4d6f-bdf6-133e2e114b5b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:15.173651 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534b3f1-b07e-4d6f-bdf6-133e2e114b5b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:15.173930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534b3f1-b07e-4d6f-bdf6-133e2e114b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:15.193685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f534b3f1-b07e-4d6f-bdf6-133e2e114b5b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:40:15.194931 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f534b3f1-b07e-4d6f-bdf6-133e2e114b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:15.204918 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1144/2288] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:40:15.209973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46338db6-4996-4f22-becf-7f0731a19c7d None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:15.221974 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46338db6-4996-4f22-becf-7f0731a19c7d None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:15.227602 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7df6a-3d38-463c-885c-8297a126deb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:15.228211 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7df6a-3d38-463c-885c-8297a126deb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:15.228533 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7df6a-3d38-463c-885c-8297a126deb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:15.228894 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7df6a-3d38-463c-885c-8297a126deb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:15.229669 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7df6a-3d38-463c-885c-8297a126deb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:40:15.230079 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7df6a-3d38-463c-885c-8297a126deb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:15.231699 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7df6a-3d38-463c-885c-8297a126deb7 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:15.244070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1145/2289] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:40:15.248479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534b3f1-b07e-4d6f-bdf6-133e2e114b5b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:15.257727 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534b3f1-b07e-4d6f-bdf6-133e2e114b5b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:15.262320 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5caa8-bb15-406e-8a0f-d82c95b41567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:15.262913 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5caa8-bb15-406e-8a0f-d82c95b41567 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:15.263193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5caa8-bb15-406e-8a0f-d82c95b41567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:15.263469 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5caa8-bb15-406e-8a0f-d82c95b41567 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:15.264362 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf5caa8-bb15-406e-8a0f-d82c95b41567 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:15.266290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf5caa8-bb15-406e-8a0f-d82c95b41567 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:15.374041 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1145/2290] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:40:15 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:40:15.378141 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7df6a-3d38-463c-885c-8297a126deb7 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:15.388425 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7df6a-3d38-463c-885c-8297a126deb7 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:15.446506 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3850e3-844e-441d-9023-23235e9658cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:15.447110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3850e3-844e-441d-9023-23235e9658cf admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:15.447390 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3850e3-844e-441d-9023-23235e9658cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:15.447668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3850e3-844e-441d-9023-23235e9658cf admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:15.448292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3850e3-844e-441d-9023-23235e9658cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:15.450261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3850e3-844e-441d-9023-23235e9658cf admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:15.539231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1146/2291] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:40:15 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:40:15.543082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5caa8-bb15-406e-8a0f-d82c95b41567 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:15.553292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5caa8-bb15-406e-8a0f-d82c95b41567 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:15.557886 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a91ea9-493c-40f8-9866-a830d78e49b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:15.558484 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a91ea9-493c-40f8-9866-a830d78e49b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:15.558766 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a91ea9-493c-40f8-9866-a830d78e49b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:15.559045 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a91ea9-493c-40f8-9866-a830d78e49b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:15.559688 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a91ea9-493c-40f8-9866-a830d78e49b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:15.561295 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a91ea9-493c-40f8-9866-a830d78e49b1 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:15.571611 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1146/2292] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:40:15.575646 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3850e3-844e-441d-9023-23235e9658cf admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:15.586248 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3850e3-844e-441d-9023-23235e9658cf admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:15.590887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55557d75-6331-429e-ab85-9d786d6e988a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:15.591477 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55557d75-6331-429e-ab85-9d786d6e988a admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:15.591756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55557d75-6331-429e-ab85-9d786d6e988a admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:15.592029 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55557d75-6331-429e-ab85-9d786d6e988a admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:15.592686 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55557d75-6331-429e-ab85-9d786d6e988a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:15.594297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55557d75-6331-429e-ab85-9d786d6e988a admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:15.604805 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1147/2293] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:40:15.610911 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a91ea9-493c-40f8-9866-a830d78e49b1 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:15.620923 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a91ea9-493c-40f8-9866-a830d78e49b1 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:15.625633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8acf2-7ec1-48f8-8f8b-70609b03fa5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:15.626265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8acf2-7ec1-48f8-8f8b-70609b03fa5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:15.626645 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8acf2-7ec1-48f8-8f8b-70609b03fa5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:15.626925 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8acf2-7ec1-48f8-8f8b-70609b03fa5b admin admin] PATH_INFO: `/v3/projects/376c4b2900354de5b06bdf3a8d2bc298/users/4c6f469725744486b03e30fec63bb271/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:15.642868 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8acf2-7ec1-48f8-8f8b-70609b03fa5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=376c4b2900354de5b06bdf3a8d2bc298, user_id=4c6f469725744486b03e30fec63bb271, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:15.645543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8acf2-7ec1-48f8-8f8b-70609b03fa5b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:15.727487 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1147/2294] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:40:15 2022] PUT /identity/v3/projects/376c4b2900354de5b06bdf3a8d2bc298/users/4c6f469725744486b03e30fec63bb271/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:40:15.738836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f424a9-a615-4250-8af3-06a5f4e7a00f None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:15.739119 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f424a9-a615-4250-8af3-06a5f4e7a00f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:15.739395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f424a9-a615-4250-8af3-06a5f4e7a00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:15.767552 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-f1f424a9-a615-4250-8af3-06a5f4e7a00f None None] MFA Rules not processed for user `4c6f469725744486b03e30fec63bb271`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:40:15.768760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-f1f424a9-a615-4250-8af3-06a5f4e7a00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:15.841559 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1148/2295] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:40:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:40:15.853419 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8acf2-7ec1-48f8-8f8b-70609b03fa5b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:15.855108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-81e8acf2-7ec1-48f8-8f8b-70609b03fa5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:15.880784 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:40:15.929603 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8acf2-7ec1-48f8-8f8b-70609b03fa5b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:15.933224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2caf7-ea6e-49d6-88b7-3cb80102eca1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:15.933829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2caf7-ea6e-49d6-88b7-3cb80102eca1 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:15.934108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2caf7-ea6e-49d6-88b7-3cb80102eca1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:15.934378 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2caf7-ea6e-49d6-88b7-3cb80102eca1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:15.942636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2caf7-ea6e-49d6-88b7-3cb80102eca1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:15.944624 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2caf7-ea6e-49d6-88b7-3cb80102eca1 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:15.953463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1148/2296] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:40:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:40:19.098168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f424a9-a615-4250-8af3-06a5f4e7a00f None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:19.121553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:40:19.166554 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f424a9-a615-4250-8af3-06a5f4e7a00f None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:19.171100 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c12f11-8852-4efe-b2f2-b658c6d2707a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:19.171727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c12f11-8852-4efe-b2f2-b658c6d2707a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:19.172052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c12f11-8852-4efe-b2f2-b658c6d2707a admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:19.172350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c12f11-8852-4efe-b2f2-b658c6d2707a admin admin] PATH_INFO: `/v3/users/4c6f469725744486b03e30fec63bb271` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:19.173516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c12f11-8852-4efe-b2f2-b658c6d2707a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c6f469725744486b03e30fec63bb271)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:19.175441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c12f11-8852-4efe-b2f2-b658c6d2707a admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:19.254949 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09dfe3-29b9-424a-a4e9-2e93a1e507be None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:19.255955 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09dfe3-29b9-424a-a4e9-2e93a1e507be None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:19.256072 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09dfe3-29b9-424a-a4e9-2e93a1e507be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:19.276250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4b09dfe3-29b9-424a-a4e9-2e93a1e507be None None] MFA Rules not processed for user `8a059ba74d744a12a4694513f8ca5789`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:40:19.285133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4b09dfe3-29b9-424a-a4e9-2e93a1e507be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:19.337150 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1149/2297] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:40:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:40:19.383745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-40c12f11-8852-4efe-b2f2-b658c6d2707a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c6f469725744486b03e30fec63bb271'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:19.493407 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-40c12f11-8852-4efe-b2f2-b658c6d2707a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c6f469725744486b03e30fec63bb271'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:19.494834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-40c12f11-8852-4efe-b2f2-b658c6d2707a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c6f469725744486b03e30fec63bb271'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:19.495450 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-40c12f11-8852-4efe-b2f2-b658c6d2707a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c6f469725744486b03e30fec63bb271'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:19.504319 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-40c12f11-8852-4efe-b2f2-b658c6d2707a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c6f469725744486b03e30fec63bb271'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:19.535368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1149/2298] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:40:19 2022] DELETE /identity/v3/users/4c6f469725744486b03e30fec63bb271 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:40:19.539681 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296676ac-d1d4-44a0-ad24-fa9f316f969b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:19.539967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296676ac-d1d4-44a0-ad24-fa9f316f969b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:19.540241 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296676ac-d1d4-44a0-ad24-fa9f316f969b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:19.560087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-296676ac-d1d4-44a0-ad24-fa9f316f969b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:40:19.561330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-296676ac-d1d4-44a0-ad24-fa9f316f969b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:19.626552 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1150/2299] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:40:19.636783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c12f11-8852-4efe-b2f2-b658c6d2707a admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:19.638590 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-40c12f11-8852-4efe-b2f2-b658c6d2707a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:19.705186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c12f11-8852-4efe-b2f2-b658c6d2707a admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:19.708769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31f403-be58-4b84-9a0a-7ef06b989d95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:19.709437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31f403-be58-4b84-9a0a-7ef06b989d95 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:19.709717 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31f403-be58-4b84-9a0a-7ef06b989d95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:19.709994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31f403-be58-4b84-9a0a-7ef06b989d95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:19.719392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31f403-be58-4b84-9a0a-7ef06b989d95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:19.721250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31f403-be58-4b84-9a0a-7ef06b989d95 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:19.731066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1150/2300] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:40:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:40:20.358767 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296676ac-d1d4-44a0-ad24-fa9f316f969b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:20.362116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-296676ac-d1d4-44a0-ad24-fa9f316f969b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:20.373140 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296676ac-d1d4-44a0-ad24-fa9f316f969b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:20.382467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc04c6-4710-4797-9677-bb2cb81ac20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:20.382467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc04c6-4710-4797-9677-bb2cb81ac20d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:20.382467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc04c6-4710-4797-9677-bb2cb81ac20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:20.382467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc04c6-4710-4797-9677-bb2cb81ac20d admin admin] PATH_INFO: `/v3/projects/376c4b2900354de5b06bdf3a8d2bc298` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:20.383030 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc04c6-4710-4797-9677-bb2cb81ac20d admin admin] RBAC: Authorizing `identity:delete_project(project_id=376c4b2900354de5b06bdf3a8d2bc298)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:20.384400 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc04c6-4710-4797-9677-bb2cb81ac20d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:20.525417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-91bc04c6-4710-4797-9677-bb2cb81ac20d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '376c4b2900354de5b06bdf3a8d2bc298', 'request_id': 'req-91bc04c6-4710-4797-9677-bb2cb81ac20d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:20.532125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-91bc04c6-4710-4797-9677-bb2cb81ac20d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '376c4b2900354de5b06bdf3a8d2bc298', 'request_id': 'req-91bc04c6-4710-4797-9677-bb2cb81ac20d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:20.539098 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-91bc04c6-4710-4797-9677-bb2cb81ac20d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '376c4b2900354de5b06bdf3a8d2bc298', 'request_id': 'req-91bc04c6-4710-4797-9677-bb2cb81ac20d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:20.540977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:40:20.636132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-91bc04c6-4710-4797-9677-bb2cb81ac20d admin admin] The token cache is being invalidate because project 376c4b2900354de5b06bdf3a8d2bc298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:40:20.636433 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-91bc04c6-4710-4797-9677-bb2cb81ac20d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:20.637515 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-91bc04c6-4710-4797-9677-bb2cb81ac20d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:40:20.639414 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1151/2301] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:40:20 2022] DELETE /identity/v3/projects/376c4b2900354de5b06bdf3a8d2bc298 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:40:20.644573 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c4731-75c8-4529-8c33-0d3be48e2a8a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:20.645213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c4731-75c8-4529-8c33-0d3be48e2a8a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:20.645677 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c4731-75c8-4529-8c33-0d3be48e2a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:20.667399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-9b4c4731-75c8-4529-8c33-0d3be48e2a8a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:40:20.668632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-9b4c4731-75c8-4529-8c33-0d3be48e2a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:20.705561 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:40:20.731524 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1151/2302] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:40:20.739546 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631c529-1041-446b-a3f6-03f63bd11fd3 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:20.739887 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631c529-1041-446b-a3f6-03f63bd11fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:20.740170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631c529-1041-446b-a3f6-03f63bd11fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:20.760364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-5631c529-1041-446b-a3f6-03f63bd11fd3 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:40:20.761603 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5631c529-1041-446b-a3f6-03f63bd11fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:20.770173 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1152/2303] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:40:20.775118 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c4731-75c8-4529-8c33-0d3be48e2a8a None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:20.785351 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c4731-75c8-4529-8c33-0d3be48e2a8a None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:20.790436 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98ea22-43e1-489e-a8b6-9ac4c6c5b7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:20.790977 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98ea22-43e1-489e-a8b6-9ac4c6c5b7d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:20.791250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98ea22-43e1-489e-a8b6-9ac4c6c5b7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:20.791519 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98ea22-43e1-489e-a8b6-9ac4c6c5b7d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:20.792161 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c98ea22-43e1-489e-a8b6-9ac4c6c5b7d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:40:20.792458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c98ea22-43e1-489e-a8b6-9ac4c6c5b7d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:20.794073 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c98ea22-43e1-489e-a8b6-9ac4c6c5b7d5 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:20.805822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1152/2304] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:40:20.810331 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631c529-1041-446b-a3f6-03f63bd11fd3 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:20.820382 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631c529-1041-446b-a3f6-03f63bd11fd3 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:20.825129 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129da02-9e9f-48a9-8683-9dc9a9fbc078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:20.826190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129da02-9e9f-48a9-8683-9dc9a9fbc078 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:20.826190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129da02-9e9f-48a9-8683-9dc9a9fbc078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:20.826383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129da02-9e9f-48a9-8683-9dc9a9fbc078 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:20.826915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129da02-9e9f-48a9-8683-9dc9a9fbc078 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:20.828810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129da02-9e9f-48a9-8683-9dc9a9fbc078 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:20.911873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1153/2305] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:40:20 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:40:20.916125 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98ea22-43e1-489e-a8b6-9ac4c6c5b7d5 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:20.926005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98ea22-43e1-489e-a8b6-9ac4c6c5b7d5 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:20.981698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bff1f-5837-4b6c-9554-a8c92365cceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:20.982298 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bff1f-5837-4b6c-9554-a8c92365cceb admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:20.982579 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bff1f-5837-4b6c-9554-a8c92365cceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:20.982853 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bff1f-5837-4b6c-9554-a8c92365cceb admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:20.983475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bff1f-5837-4b6c-9554-a8c92365cceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:20.985425 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bff1f-5837-4b6c-9554-a8c92365cceb admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:21.088501 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1153/2306] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:40:20 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:40:21.092573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129da02-9e9f-48a9-8683-9dc9a9fbc078 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:21.103740 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129da02-9e9f-48a9-8683-9dc9a9fbc078 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:21.108451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739baf8-c252-4c93-aa4d-4e11836c36e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:21.119133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739baf8-c252-4c93-aa4d-4e11836c36e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:21.119133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739baf8-c252-4c93-aa4d-4e11836c36e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:21.119133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739baf8-c252-4c93-aa4d-4e11836c36e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:21.119133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739baf8-c252-4c93-aa4d-4e11836c36e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:21.119133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739baf8-c252-4c93-aa4d-4e11836c36e1 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:21.122832 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1154/2307] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:40:21.127048 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bff1f-5837-4b6c-9554-a8c92365cceb admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:21.137936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bff1f-5837-4b6c-9554-a8c92365cceb admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:21.144605 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9e482-9996-4eed-a9e6-94b7c9e731a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:21.148451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9e482-9996-4eed-a9e6-94b7c9e731a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:21.148451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9e482-9996-4eed-a9e6-94b7c9e731a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:21.148451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9e482-9996-4eed-a9e6-94b7c9e731a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:21.148451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9e482-9996-4eed-a9e6-94b7c9e731a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:21.148451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9e482-9996-4eed-a9e6-94b7c9e731a9 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:21.160450 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1154/2308] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:40:21.164739 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739baf8-c252-4c93-aa4d-4e11836c36e1 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:21.180214 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739baf8-c252-4c93-aa4d-4e11836c36e1 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:21.184835 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129c438-f1c9-42d0-8087-a86b0b90af1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:21.185530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129c438-f1c9-42d0-8087-a86b0b90af1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:21.185809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129c438-f1c9-42d0-8087-a86b0b90af1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:21.186082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129c438-f1c9-42d0-8087-a86b0b90af1a admin admin] PATH_INFO: `/v3/projects/b082859e089143dda3eefee066e4916d/users/884620036bb44462964db97238091c19/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:21.203722 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129c438-f1c9-42d0-8087-a86b0b90af1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b082859e089143dda3eefee066e4916d, user_id=884620036bb44462964db97238091c19, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:21.206793 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129c438-f1c9-42d0-8087-a86b0b90af1a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:21.277450 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1155/2309] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:40:21 2022] PUT /identity/v3/projects/b082859e089143dda3eefee066e4916d/users/884620036bb44462964db97238091c19/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:40:21.282929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c400f5-0a59-44e8-9638-0deaf4b64b1a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:21.283215 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c400f5-0a59-44e8-9638-0deaf4b64b1a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:21.283494 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c400f5-0a59-44e8-9638-0deaf4b64b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:21.302998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-f6c400f5-0a59-44e8-9638-0deaf4b64b1a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:40:21.304199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-f6c400f5-0a59-44e8-9638-0deaf4b64b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:21.362409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1155/2310] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:40:21.372265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129c438-f1c9-42d0-8087-a86b0b90af1a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:21.374194 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a129c438-f1c9-42d0-8087-a86b0b90af1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:21.429823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129c438-f1c9-42d0-8087-a86b0b90af1a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:21.433465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267a835-f4ed-4a67-9b65-b5988aacf1f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:21.434077 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267a835-f4ed-4a67-9b65-b5988aacf1f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:21.434338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267a835-f4ed-4a67-9b65-b5988aacf1f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:21.434611 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267a835-f4ed-4a67-9b65-b5988aacf1f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:21.444034 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267a835-f4ed-4a67-9b65-b5988aacf1f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:21.445490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267a835-f4ed-4a67-9b65-b5988aacf1f7 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:21.454655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1156/2311] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:40:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:40:22.846782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c427a-9237-4104-a12f-a2ba904196dd None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:22.847301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c427a-9237-4104-a12f-a2ba904196dd None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:22.847459 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c427a-9237-4104-a12f-a2ba904196dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:22.876488 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-5f8c427a-9237-4104-a12f-a2ba904196dd None None] MFA Rules not processed for user `884620036bb44462964db97238091c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:40:22.877781 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5f8c427a-9237-4104-a12f-a2ba904196dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:22.912676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:40:22.942242 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1156/2312] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:40:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:40:22.952990 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267a835-f4ed-4a67-9b65-b5988aacf1f7 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:40:22.954423 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f267a835-f4ed-4a67-9b65-b5988aacf1f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:23.010639 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267a835-f4ed-4a67-9b65-b5988aacf1f7 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:40:23.014479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7546bfe-d11d-4d47-abd2-5c12b50d8377 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:40:23.015076 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7546bfe-d11d-4d47-abd2-5c12b50d8377 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:23.015357 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7546bfe-d11d-4d47-abd2-5c12b50d8377 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:23.015641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7546bfe-d11d-4d47-abd2-5c12b50d8377 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:23.024383 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7546bfe-d11d-4d47-abd2-5c12b50d8377 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:40:23.025825 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7546bfe-d11d-4d47-abd2-5c12b50d8377 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:40:23.034683 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1157/2313] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:40:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:40:24.852460 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab3870-21d7-4b44-84ed-690ff94614e8 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:24.853004 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab3870-21d7-4b44-84ed-690ff94614e8 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:24.853004 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab3870-21d7-4b44-84ed-690ff94614e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:24.872281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-79ab3870-21d7-4b44-84ed-690ff94614e8 None None] MFA Rules not processed for user `884620036bb44462964db97238091c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:40:24.873677 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-79ab3870-21d7-4b44-84ed-690ff94614e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:24.880844 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1157/2314] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:40:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:40:33.030693 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=81 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:40:34.866677 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=14 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:40:50.335226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025eee4-f5a3-4bda-b046-2176cbbfcc01 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:40:50.335821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025eee4-f5a3-4bda-b046-2176cbbfcc01 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:40:50.335821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025eee4-f5a3-4bda-b046-2176cbbfcc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:40:50.355969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1025eee4-f5a3-4bda-b046-2176cbbfcc01 None None] MFA Rules not processed for user `d4d9bea213fa4401992d8e7894726e05`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:40:50.357108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1025eee4-f5a3-4bda-b046-2176cbbfcc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:40:50.417007 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1158/2315] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:40:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:41:00.409533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=25 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:41:18.449441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab3870-21d7-4b44-84ed-690ff94614e8 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:41:18.450871 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-79ab3870-21d7-4b44-84ed-690ff94614e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:41:18.463077 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab3870-21d7-4b44-84ed-690ff94614e8 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:41:18.466653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8e967-773a-4250-8d2b-5ac19125b569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:41:18.467239 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8e967-773a-4250-8d2b-5ac19125b569 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:41:18.467511 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8e967-773a-4250-8d2b-5ac19125b569 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:41:18.467779 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8e967-773a-4250-8d2b-5ac19125b569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:41:18.469558 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-76a8e967-773a-4250-8d2b-5ac19125b569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:41:18.546309 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8e967-773a-4250-8d2b-5ac19125b569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:41:18.547804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8e967-773a-4250-8d2b-5ac19125b569 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:41:18.561021 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1158/2316] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:41:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:41:20.129329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025eee4-f5a3-4bda-b046-2176cbbfcc01 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:41:20.139580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025eee4-f5a3-4bda-b046-2176cbbfcc01 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:41:20.143462 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09bf87-3f08-42dd-b360-cefaec92f96e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:41:20.144053 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09bf87-3f08-42dd-b360-cefaec92f96e service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:41:20.144335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09bf87-3f08-42dd-b360-cefaec92f96e service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:41:20.144604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09bf87-3f08-42dd-b360-cefaec92f96e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:41:20.147138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-df09bf87-3f08-42dd-b360-cefaec92f96e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:41:20.160025 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09bf87-3f08-42dd-b360-cefaec92f96e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:41:20.161690 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09bf87-3f08-42dd-b360-cefaec92f96e service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:41:20.172488 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1159/2317] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:41:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:41:28.555028 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=34 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:41:29.825429 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7718f37-1fba-4639-bac6-ef7bc9418647 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:41:29.826044 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7718f37-1fba-4639-bac6-ef7bc9418647 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:41:29.826044 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7718f37-1fba-4639-bac6-ef7bc9418647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:41:29.845563 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-e7718f37-1fba-4639-bac6-ef7bc9418647 None None] MFA Rules not processed for user `d3235496c38249a388e4afc263eadbc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:41:29.846747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e7718f37-1fba-4639-bac6-ef7bc9418647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:41:29.901296 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1159/2318] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:41:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:41:30.167651 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=6 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:41:33.489478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09bf87-3f08-42dd-b360-cefaec92f96e service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:41:33.497986 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09bf87-3f08-42dd-b360-cefaec92f96e service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:41:33.501890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e619f-74c1-4d4c-aa7d-3b9b51fcb519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:41:33.502512 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e619f-74c1-4d4c-aa7d-3b9b51fcb519 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:41:33.502785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e619f-74c1-4d4c-aa7d-3b9b51fcb519 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:41:33.503050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e619f-74c1-4d4c-aa7d-3b9b51fcb519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:41:33.504812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-621e619f-74c1-4d4c-aa7d-3b9b51fcb519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:41:33.564131 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e619f-74c1-4d4c-aa7d-3b9b51fcb519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:41:33.565581 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e619f-74c1-4d4c-aa7d-3b9b51fcb519 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:41:33.575099 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1160/2319] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:41:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:41:34.880563 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc51e53-efa4-4798-9b26-3fd7583be05b None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:41:34.881125 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc51e53-efa4-4798-9b26-3fd7583be05b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:41:34.881225 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc51e53-efa4-4798-9b26-3fd7583be05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:41:34.901905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-abc51e53-efa4-4798-9b26-3fd7583be05b None None] MFA Rules not processed for user `6712334ebb77462daa89be58d4480846`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:41:34.903102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-abc51e53-efa4-4798-9b26-3fd7583be05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:41:34.910697 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1160/2320] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:41:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:41:43.078327 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e619f-74c1-4d4c-aa7d-3b9b51fcb519 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:41:43.079815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-621e619f-74c1-4d4c-aa7d-3b9b51fcb519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:41:43.095703 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e619f-74c1-4d4c-aa7d-3b9b51fcb519 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:41:43.101502 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1807935-1ecf-4a83-8baf-2221782ca94b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:41:43.102095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1807935-1ecf-4a83-8baf-2221782ca94b service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:41:43.102367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1807935-1ecf-4a83-8baf-2221782ca94b service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:41:43.102635 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1807935-1ecf-4a83-8baf-2221782ca94b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:41:43.104291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b1807935-1ecf-4a83-8baf-2221782ca94b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:41:43.166152 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1807935-1ecf-4a83-8baf-2221782ca94b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:41:43.167975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1807935-1ecf-4a83-8baf-2221782ca94b service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:41:43.178410 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1161/2321] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:41:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:41:44.895733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=28 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:41:53.149268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52808c21-0e6b-4444-8024-d096d1dddcbf None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:41:53.149896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52808c21-0e6b-4444-8024-d096d1dddcbf None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:41:53.149896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52808c21-0e6b-4444-8024-d096d1dddcbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:41:53.169558 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-52808c21-0e6b-4444-8024-d096d1dddcbf None None] MFA Rules not processed for user `884620036bb44462964db97238091c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:41:53.170746 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-52808c21-0e6b-4444-8024-d096d1dddcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:41:53.173601 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=56 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:41:53.178362 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1161/2322] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:41:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:41:53.448821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1807935-1ecf-4a83-8baf-2221782ca94b service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:41:53.461359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1807935-1ecf-4a83-8baf-2221782ca94b service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:41:53.466463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821522f4-5598-43a0-8030-5eb1c4ce80e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:41:53.467307 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821522f4-5598-43a0-8030-5eb1c4ce80e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:41:53.467716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821522f4-5598-43a0-8030-5eb1c4ce80e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:41:53.468061 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821522f4-5598-43a0-8030-5eb1c4ce80e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:41:53.470267 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-821522f4-5598-43a0-8030-5eb1c4ce80e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:41:53.479668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821522f4-5598-43a0-8030-5eb1c4ce80e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:41:53.481097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821522f4-5598-43a0-8030-5eb1c4ce80e0 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:41:53.489162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1162/2323] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:41:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:41:53.883303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52808c21-0e6b-4444-8024-d096d1dddcbf None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:41:53.884735 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-52808c21-0e6b-4444-8024-d096d1dddcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:41:53.966777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52808c21-0e6b-4444-8024-d096d1dddcbf None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:41:53.972253 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ebefb-9dfe-45ff-a1ff-522e046d0498 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0ef47a60f54ca1853fd267e308c58b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:41:53.972970 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ebefb-9dfe-45ff-a1ff-522e046d0498 service placement] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:41:53.973341 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ebefb-9dfe-45ff-a1ff-522e046d0498 service placement] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:41:53.973654 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ebefb-9dfe-45ff-a1ff-522e046d0498 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:41:53.976151 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-dd5ebefb-9dfe-45ff-a1ff-522e046d0498 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:41:53.988823 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ebefb-9dfe-45ff-a1ff-522e046d0498 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:41:53.991291 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ebefb-9dfe-45ff-a1ff-522e046d0498 service placement] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:41:54.002605 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1162/2324] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:41:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:41:54.686959 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821522f4-5598-43a0-8030-5eb1c4ce80e0 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:41:54.697239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821522f4-5598-43a0-8030-5eb1c4ce80e0 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:41:54.701253 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be4702-54c0-4d75-9589-f388ba6bb471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:41:54.706812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be4702-54c0-4d75-9589-f388ba6bb471 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:41:54.706812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be4702-54c0-4d75-9589-f388ba6bb471 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:41:54.706812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be4702-54c0-4d75-9589-f388ba6bb471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:41:54.706812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e8be4702-54c0-4d75-9589-f388ba6bb471 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:41:54.720611 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be4702-54c0-4d75-9589-f388ba6bb471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:41:54.722353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be4702-54c0-4d75-9589-f388ba6bb471 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:41:54.732071 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1163/2325] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:41:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:41:58.791857 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ebefb-9dfe-45ff-a1ff-522e046d0498 service placement] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:41:58.793362 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-dd5ebefb-9dfe-45ff-a1ff-522e046d0498 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:41:58.813107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ebefb-9dfe-45ff-a1ff-522e046d0498 service placement] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:41:58.832023 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c36d96-57f9-46b3-a567-9fda29639984 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:41:58.832023 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c36d96-57f9-46b3-a567-9fda29639984 service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:41:58.832023 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c36d96-57f9-46b3-a567-9fda29639984 service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:41:58.832023 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c36d96-57f9-46b3-a567-9fda29639984 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:41:58.834209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-f3c36d96-57f9-46b3-a567-9fda29639984 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:41:58.850789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c36d96-57f9-46b3-a567-9fda29639984 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:41:58.852610 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c36d96-57f9-46b3-a567-9fda29639984 service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:41:58.867778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1163/2326] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:41:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:42:04.728579 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=12 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:42:06.166545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be4702-54c0-4d75-9589-f388ba6bb471 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:42:06.182131 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be4702-54c0-4d75-9589-f388ba6bb471 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:42:06.188444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb6798-2593-40f6-9af1-ebe74ae34f48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:42:06.189204 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb6798-2593-40f6-9af1-ebe74ae34f48 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:42:06.189562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb6798-2593-40f6-9af1-ebe74ae34f48 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:42:06.189855 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb6798-2593-40f6-9af1-ebe74ae34f48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:42:06.191860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0aeb6798-2593-40f6-9af1-ebe74ae34f48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:42:06.203901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb6798-2593-40f6-9af1-ebe74ae34f48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:42:06.205850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb6798-2593-40f6-9af1-ebe74ae34f48 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:42:06.216903 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1164/2327] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:42:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:42:07.631377 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c36d96-57f9-46b3-a567-9fda29639984 service nova] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:42:07.643515 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c36d96-57f9-46b3-a567-9fda29639984 service nova] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:42:07.649759 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e0f0a-ac25-480e-b093-13cc28cfeb5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:42:07.650365 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e0f0a-ac25-480e-b093-13cc28cfeb5e service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:42:07.650645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e0f0a-ac25-480e-b093-13cc28cfeb5e service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:42:07.650920 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e0f0a-ac25-480e-b093-13cc28cfeb5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:42:07.652654 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-0f4e0f0a-ac25-480e-b093-13cc28cfeb5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:42:07.665918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e0f0a-ac25-480e-b093-13cc28cfeb5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:42:07.667497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e0f0a-ac25-480e-b093-13cc28cfeb5e service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:42:07.686482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1164/2328] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:42:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:42:11.260977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33329985-75f6-4976-8537-763ba9f32c20 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:42:11.261491 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33329985-75f6-4976-8537-763ba9f32c20 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:42:11.262683 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33329985-75f6-4976-8537-763ba9f32c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:42:11.280952 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-33329985-75f6-4976-8537-763ba9f32c20 None None] MFA Rules not processed for user `4ae237601b4a4aab9e969fedb5ee37d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:42:11.282281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-33329985-75f6-4976-8537-763ba9f32c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:42:11.350517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1165/2329] 10.222.0.42 () {58 vars in 967 bytes} [Mon Sep 19 10:42:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:42:11.478845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e0f0a-ac25-480e-b093-13cc28cfeb5e service nova] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:42:11.480455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-0f4e0f0a-ac25-480e-b093-13cc28cfeb5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:42:11.556965 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e0f0a-ac25-480e-b093-13cc28cfeb5e service nova] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:42:11.561973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d518b4-9658-4671-af25-05159d079fee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17a8bc6c506c45be897bda23eaab485a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:42:11.562612 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d518b4-9658-4671-af25-05159d079fee service swift] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:42:11.562894 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d518b4-9658-4671-af25-05159d079fee service swift] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:42:11.563166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d518b4-9658-4671-af25-05159d079fee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:42:11.571335 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d518b4-9658-4671-af25-05159d079fee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:42:11.574213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d518b4-9658-4671-af25-05159d079fee service swift] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:42:11.585228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1165/2330] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 19 10:42:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 10:42:14.955987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b09d97-f3f7-41ac-9d59-d53c3a923dbe None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:42:14.956539 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b09d97-f3f7-41ac-9d59-d53c3a923dbe None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:42:14.956720 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b09d97-f3f7-41ac-9d59-d53c3a923dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:42:14.979631 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-29b09d97-f3f7-41ac-9d59-d53c3a923dbe None None] MFA Rules not processed for user `8a059ba74d744a12a4694513f8ca5789`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:42:14.980872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-29b09d97-f3f7-41ac-9d59-d53c3a923dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:42:14.988717 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1166/2331] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:42:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:42:21.578458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=79 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:42:22.075747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d518b4-9658-4671-af25-05159d079fee service swift] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:42:22.088138 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d518b4-9658-4671-af25-05159d079fee service swift] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:42:22.093239 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6eaed2-697e-43ce-8a83-d02a15165dbb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0ef47a60f54ca1853fd267e308c58b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:42:22.093833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6eaed2-697e-43ce-8a83-d02a15165dbb service placement] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:42:22.094111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6eaed2-697e-43ce-8a83-d02a15165dbb service placement] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:42:22.094386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6eaed2-697e-43ce-8a83-d02a15165dbb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:42:22.096091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-ed6eaed2-697e-43ce-8a83-d02a15165dbb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:42:22.108850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6eaed2-697e-43ce-8a83-d02a15165dbb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:42:22.110697 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6eaed2-697e-43ce-8a83-d02a15165dbb service placement] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:42:22.120918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1166/2332] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:42:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:42:22.269755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b09d97-f3f7-41ac-9d59-d53c3a923dbe None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:42:22.269755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-29b09d97-f3f7-41ac-9d59-d53c3a923dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:42:22.269755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b09d97-f3f7-41ac-9d59-d53c3a923dbe None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:42:22.271540 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635d299-dc90-4d7c-9a28-36e8d79d2d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:42:22.271540 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635d299-dc90-4d7c-9a28-36e8d79d2d41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:42:22.271540 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635d299-dc90-4d7c-9a28-36e8d79d2d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:42:22.271540 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635d299-dc90-4d7c-9a28-36e8d79d2d41 admin admin] PATH_INFO: `/v3/users/0b3fcdf3a12147f591c0ea67b57c8c7e` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:42:22.272526 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6635d299-dc90-4d7c-9a28-36e8d79d2d41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b3fcdf3a12147f591c0ea67b57c8c7e)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:42:22.275696 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6635d299-dc90-4d7c-9a28-36e8d79d2d41 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:42:22.515746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6635d299-dc90-4d7c-9a28-36e8d79d2d41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b3fcdf3a12147f591c0ea67b57c8c7e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:22.516854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6635d299-dc90-4d7c-9a28-36e8d79d2d41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b3fcdf3a12147f591c0ea67b57c8c7e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:22.648602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6635d299-dc90-4d7c-9a28-36e8d79d2d41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b3fcdf3a12147f591c0ea67b57c8c7e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:22.649293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6635d299-dc90-4d7c-9a28-36e8d79d2d41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b3fcdf3a12147f591c0ea67b57c8c7e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:22.665170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6635d299-dc90-4d7c-9a28-36e8d79d2d41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b3fcdf3a12147f591c0ea67b57c8c7e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:22.682034 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1167/2333] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:42:22 2022] DELETE /identity/v3/users/0b3fcdf3a12147f591c0ea67b57c8c7e => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:42:22.978266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6eaed2-697e-43ce-8a83-d02a15165dbb service placement] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:42:22.989870 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6eaed2-697e-43ce-8a83-d02a15165dbb service placement] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:42:23.057100 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca131b-ffe6-4ad2-8b9d-381703dd99fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:42:23.057723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca131b-ffe6-4ad2-8b9d-381703dd99fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:42:23.058009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca131b-ffe6-4ad2-8b9d-381703dd99fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:42:23.058285 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca131b-ffe6-4ad2-8b9d-381703dd99fb admin admin] PATH_INFO: `/v3/projects/c84c95680abc4f28b49cd092ca96fe32` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:42:23.059543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca131b-ffe6-4ad2-8b9d-381703dd99fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c84c95680abc4f28b49cd092ca96fe32)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:42:23.061663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca131b-ffe6-4ad2-8b9d-381703dd99fb admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:42:23.191244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-56ca131b-ffe6-4ad2-8b9d-381703dd99fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c84c95680abc4f28b49cd092ca96fe32', 'request_id': 'req-56ca131b-ffe6-4ad2-8b9d-381703dd99fb'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:23.197332 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-56ca131b-ffe6-4ad2-8b9d-381703dd99fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c84c95680abc4f28b49cd092ca96fe32', 'request_id': 'req-56ca131b-ffe6-4ad2-8b9d-381703dd99fb'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:23.203435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-56ca131b-ffe6-4ad2-8b9d-381703dd99fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c84c95680abc4f28b49cd092ca96fe32', 'request_id': 'req-56ca131b-ffe6-4ad2-8b9d-381703dd99fb'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:23.271347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-56ca131b-ffe6-4ad2-8b9d-381703dd99fb admin admin] The token cache is being invalidate because project c84c95680abc4f28b49cd092ca96fe32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:42:23.271637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-56ca131b-ffe6-4ad2-8b9d-381703dd99fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:23.272282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-56ca131b-ffe6-4ad2-8b9d-381703dd99fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:23.273838 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1167/2334] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:42:22 2022] DELETE /identity/v3/projects/c84c95680abc4f28b49cd092ca96fe32 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:42:23.278048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635d299-dc90-4d7c-9a28-36e8d79d2d41 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:42:23.281810 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6635d299-dc90-4d7c-9a28-36e8d79d2d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:42:23.295316 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635d299-dc90-4d7c-9a28-36e8d79d2d41 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:42:23.353806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:42:23.360406 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047eef34-d5e5-45b6-a30c-b77aff158dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:42:23.366268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047eef34-d5e5-45b6-a30c-b77aff158dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:42:23.366268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047eef34-d5e5-45b6-a30c-b77aff158dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:42:23.366268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047eef34-d5e5-45b6-a30c-b77aff158dc9 admin admin] PATH_INFO: `/v3/users/75a2146ad007405089cfcd83be256ebd` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:42:23.366268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047eef34-d5e5-45b6-a30c-b77aff158dc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75a2146ad007405089cfcd83be256ebd)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:42:23.366268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047eef34-d5e5-45b6-a30c-b77aff158dc9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:42:23.534204 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-047eef34-d5e5-45b6-a30c-b77aff158dc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75a2146ad007405089cfcd83be256ebd'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:23.536328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-047eef34-d5e5-45b6-a30c-b77aff158dc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75a2146ad007405089cfcd83be256ebd'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:23.598255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-047eef34-d5e5-45b6-a30c-b77aff158dc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75a2146ad007405089cfcd83be256ebd'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:23.599108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-047eef34-d5e5-45b6-a30c-b77aff158dc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75a2146ad007405089cfcd83be256ebd'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:23.613955 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-047eef34-d5e5-45b6-a30c-b77aff158dc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75a2146ad007405089cfcd83be256ebd'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:23.630962 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1168/2335] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:42:23 2022] DELETE /identity/v3/users/75a2146ad007405089cfcd83be256ebd => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:42:23.960485 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca131b-ffe6-4ad2-8b9d-381703dd99fb admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:42:23.962360 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-56ca131b-ffe6-4ad2-8b9d-381703dd99fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:42:23.979928 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca131b-ffe6-4ad2-8b9d-381703dd99fb admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:42:24.051457 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380af72c-66fe-4fac-9e5f-7cab1b19c95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:42:24.052762 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380af72c-66fe-4fac-9e5f-7cab1b19c95a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:42:24.053146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380af72c-66fe-4fac-9e5f-7cab1b19c95a admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:42:24.053464 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380af72c-66fe-4fac-9e5f-7cab1b19c95a admin admin] PATH_INFO: `/v3/projects/1ced0349fde44cbda660d1dbddb61162` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:42:24.054800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380af72c-66fe-4fac-9e5f-7cab1b19c95a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ced0349fde44cbda660d1dbddb61162)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:42:24.057172 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380af72c-66fe-4fac-9e5f-7cab1b19c95a admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:42:24.082875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:42:24.180394 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-380af72c-66fe-4fac-9e5f-7cab1b19c95a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ced0349fde44cbda660d1dbddb61162', 'request_id': 'req-380af72c-66fe-4fac-9e5f-7cab1b19c95a'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:24.186649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-380af72c-66fe-4fac-9e5f-7cab1b19c95a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ced0349fde44cbda660d1dbddb61162', 'request_id': 'req-380af72c-66fe-4fac-9e5f-7cab1b19c95a'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:24.193269 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-380af72c-66fe-4fac-9e5f-7cab1b19c95a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ced0349fde44cbda660d1dbddb61162', 'request_id': 'req-380af72c-66fe-4fac-9e5f-7cab1b19c95a'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:24.271925 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-380af72c-66fe-4fac-9e5f-7cab1b19c95a admin admin] The token cache is being invalidate because project 1ced0349fde44cbda660d1dbddb61162 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:42:24.272120 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-380af72c-66fe-4fac-9e5f-7cab1b19c95a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:24.272748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-380af72c-66fe-4fac-9e5f-7cab1b19c95a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:24.274278 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1168/2336] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:42:23 2022] DELETE /identity/v3/projects/1ced0349fde44cbda660d1dbddb61162 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:42:33.624869 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=27,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:42:34.267668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:42:34.326483 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1399194-6cfe-4498-ab89-02a189805f74 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:42:34.326764 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1399194-6cfe-4498-ab89-02a189805f74 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:42:34.327133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1399194-6cfe-4498-ab89-02a189805f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:42:34.347504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f1399194-6cfe-4498-ab89-02a189805f74 None None] MFA Rules not processed for user `4ae237601b4a4aab9e969fedb5ee37d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:42:34.348698 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f1399194-6cfe-4498-ab89-02a189805f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:42:34.414180 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1169/2337] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 19 10:42:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:42:34.435729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380af72c-66fe-4fac-9e5f-7cab1b19c95a admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:42:34.437823 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-380af72c-66fe-4fac-9e5f-7cab1b19c95a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:42:34.526185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380af72c-66fe-4fac-9e5f-7cab1b19c95a admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:42:34.531555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376a80e-4fd1-43fd-a915-9a45de841673 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17a8bc6c506c45be897bda23eaab485a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:42:34.532273 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376a80e-4fd1-43fd-a915-9a45de841673 service swift] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:42:34.532670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376a80e-4fd1-43fd-a915-9a45de841673 service swift] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:42:34.533041 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376a80e-4fd1-43fd-a915-9a45de841673 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:42:34.542410 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376a80e-4fd1-43fd-a915-9a45de841673 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:42:34.544067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376a80e-4fd1-43fd-a915-9a45de841673 service swift] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:42:34.554112 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1169/2338] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 19 10:42:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 10:42:36.451058 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1399194-6cfe-4498-ab89-02a189805f74 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:42:36.452786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f1399194-6cfe-4498-ab89-02a189805f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:42:36.593466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1399194-6cfe-4498-ab89-02a189805f74 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:42:36.600340 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b1dfa-2147-4442-a22e-fb921a777fc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:42:36.601183 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b1dfa-2147-4442-a22e-fb921a777fc3 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:42:36.601624 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b1dfa-2147-4442-a22e-fb921a777fc3 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:42:36.602093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b1dfa-2147-4442-a22e-fb921a777fc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:42:36.604690 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-044b1dfa-2147-4442-a22e-fb921a777fc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:42:36.625860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b1dfa-2147-4442-a22e-fb921a777fc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:42:36.627778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b1dfa-2147-4442-a22e-fb921a777fc3 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:42:36.645888 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1170/2339] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:42:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:42:45.057451 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=32 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:42:46.636308 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=59 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:42:52.858363 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376a80e-4fd1-43fd-a915-9a45de841673 service swift] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:42:52.864438 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-8376a80e-4fd1-43fd-a915-9a45de841673 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:42:52.938160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376a80e-4fd1-43fd-a915-9a45de841673 service swift] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:42:52.946346 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95b873-6f27-47e4-97a3-119bb7beee40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:42:52.947228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95b873-6f27-47e4-97a3-119bb7beee40 service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:42:52.947647 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95b873-6f27-47e4-97a3-119bb7beee40 service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:42:52.948065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95b873-6f27-47e4-97a3-119bb7beee40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:42:52.950687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-3e95b873-6f27-47e4-97a3-119bb7beee40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:42:53.020026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95b873-6f27-47e4-97a3-119bb7beee40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:42:53.022264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95b873-6f27-47e4-97a3-119bb7beee40 service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:42:53.031848 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1170/2340] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:42:52 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:42:55.340086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b1dfa-2147-4442-a22e-fb921a777fc3 service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:42:55.341605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-044b1dfa-2147-4442-a22e-fb921a777fc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:42:55.461083 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b1dfa-2147-4442-a22e-fb921a777fc3 service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:42:55.464721 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ab565-98ed-49ab-b453-cbf970bd32cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:42:55.465359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ab565-98ed-49ab-b453-cbf970bd32cf service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:42:55.465636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ab565-98ed-49ab-b453-cbf970bd32cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:42:55.465909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ab565-98ed-49ab-b453-cbf970bd32cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:42:55.467681 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a74ab565-98ed-49ab-b453-cbf970bd32cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:42:55.559255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ab565-98ed-49ab-b453-cbf970bd32cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:42:55.560692 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ab565-98ed-49ab-b453-cbf970bd32cf service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:42:55.581986 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1171/2341] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:42:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:42:59.502552 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95b873-6f27-47e4-97a3-119bb7beee40 service nova] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:42:59.504138 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-3e95b873-6f27-47e4-97a3-119bb7beee40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:42:59.518039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95b873-6f27-47e4-97a3-119bb7beee40 service nova] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:42:59.523574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057453e2-d847-49c9-8682-949bdffa4dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:42:59.524438 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057453e2-d847-49c9-8682-949bdffa4dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:42:59.524719 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057453e2-d847-49c9-8682-949bdffa4dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:42:59.524994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057453e2-d847-49c9-8682-949bdffa4dce admin admin] PATH_INFO: `/v3/users/d3235496c38249a388e4afc263eadbc3` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:42:59.526163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057453e2-d847-49c9-8682-949bdffa4dce admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3235496c38249a388e4afc263eadbc3)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:42:59.528088 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057453e2-d847-49c9-8682-949bdffa4dce admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:42:59.709698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-057453e2-d847-49c9-8682-949bdffa4dce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3235496c38249a388e4afc263eadbc3'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:59.798480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-057453e2-d847-49c9-8682-949bdffa4dce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3235496c38249a388e4afc263eadbc3'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:59.799112 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-057453e2-d847-49c9-8682-949bdffa4dce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3235496c38249a388e4afc263eadbc3'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:59.799749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-057453e2-d847-49c9-8682-949bdffa4dce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3235496c38249a388e4afc263eadbc3'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:59.807840 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-057453e2-d847-49c9-8682-949bdffa4dce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3235496c38249a388e4afc263eadbc3'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:42:59.833951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1171/2342] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:42:59 2022] DELETE /identity/v3/users/d3235496c38249a388e4afc263eadbc3 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:00.160653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ab565-98ed-49ab-b453-cbf970bd32cf service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:00.173883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ab565-98ed-49ab-b453-cbf970bd32cf service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:00.204504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:00.235391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013b40e-e8b5-456f-bbdd-c155364309ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:00.236027 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013b40e-e8b5-456f-bbdd-c155364309ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:00.236304 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013b40e-e8b5-456f-bbdd-c155364309ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:00.236581 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013b40e-e8b5-456f-bbdd-c155364309ea admin admin] PATH_INFO: `/v3/projects/af388d6e63d641528e8f00de41c59490` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:00.237737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013b40e-e8b5-456f-bbdd-c155364309ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=af388d6e63d641528e8f00de41c59490)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:00.239663 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013b40e-e8b5-456f-bbdd-c155364309ea admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:00.368861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0013b40e-e8b5-456f-bbdd-c155364309ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af388d6e63d641528e8f00de41c59490', 'request_id': 'req-0013b40e-e8b5-456f-bbdd-c155364309ea'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:00.374483 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0013b40e-e8b5-456f-bbdd-c155364309ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af388d6e63d641528e8f00de41c59490', 'request_id': 'req-0013b40e-e8b5-456f-bbdd-c155364309ea'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:00.381512 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0013b40e-e8b5-456f-bbdd-c155364309ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af388d6e63d641528e8f00de41c59490', 'request_id': 'req-0013b40e-e8b5-456f-bbdd-c155364309ea'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:00.466185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0013b40e-e8b5-456f-bbdd-c155364309ea admin admin] The token cache is being invalidate because project af388d6e63d641528e8f00de41c59490 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:43:00.466581 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0013b40e-e8b5-456f-bbdd-c155364309ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:00.467115 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-0013b40e-e8b5-456f-bbdd-c155364309ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:00.468615 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1172/2343] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:43:00 2022] DELETE /identity/v3/projects/af388d6e63d641528e8f00de41c59490 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:00.475376 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2bb8d-1885-43c8-9c88-5dd8346462ed None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:00.475657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2bb8d-1885-43c8-9c88-5dd8346462ed None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:00.475931 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2bb8d-1885-43c8-9c88-5dd8346462ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:00.509926 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:00.544811 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-62b2bb8d-1885-43c8-9c88-5dd8346462ed None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:00.546153 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-62b2bb8d-1885-43c8-9c88-5dd8346462ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:00.618935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1172/2344] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:43:00.628470 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1f437-70c9-4d0e-a848-27679d64faea None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:00.628854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1f437-70c9-4d0e-a848-27679d64faea None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:00.632101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1f437-70c9-4d0e-a848-27679d64faea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:00.659632 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-31f1f437-70c9-4d0e-a848-27679d64faea None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:00.659632 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-31f1f437-70c9-4d0e-a848-27679d64faea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:00.667027 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1173/2345] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:00.671472 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2bb8d-1885-43c8-9c88-5dd8346462ed None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:00.683251 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2bb8d-1885-43c8-9c88-5dd8346462ed None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:00.688191 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e5085-3c83-4082-896a-31042794850b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:00.688809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e5085-3c83-4082-896a-31042794850b admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:00.689097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e5085-3c83-4082-896a-31042794850b admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:00.689373 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e5085-3c83-4082-896a-31042794850b admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:00.690031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e5085-3c83-4082-896a-31042794850b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:43:00.690337 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e5085-3c83-4082-896a-31042794850b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:00.693520 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e5085-3c83-4082-896a-31042794850b admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:00.721280 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1173/2346] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:43:00.725333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1f437-70c9-4d0e-a848-27679d64faea None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:00.735514 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1f437-70c9-4d0e-a848-27679d64faea None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:00.745338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38338779-9d77-49fb-b031-511f45d3e418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:00.746217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38338779-9d77-49fb-b031-511f45d3e418 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:00.746503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38338779-9d77-49fb-b031-511f45d3e418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:00.746779 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38338779-9d77-49fb-b031-511f45d3e418 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:00.747422 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38338779-9d77-49fb-b031-511f45d3e418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:00.749361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38338779-9d77-49fb-b031-511f45d3e418 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:00.905127 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1174/2347] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:43:00 2022] POST /identity/v3/projects => generated 385 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:43:00.910887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e5085-3c83-4082-896a-31042794850b admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:00.937369 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e5085-3c83-4082-896a-31042794850b admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:01.015868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd308993-ebd6-4fd5-a982-4cfde7c80c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:01.016656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd308993-ebd6-4fd5-a982-4cfde7c80c4e admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:01.016831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd308993-ebd6-4fd5-a982-4cfde7c80c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:01.017151 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd308993-ebd6-4fd5-a982-4cfde7c80c4e admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:01.017874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd308993-ebd6-4fd5-a982-4cfde7c80c4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:01.020175 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd308993-ebd6-4fd5-a982-4cfde7c80c4e admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:01.117663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1174/2348] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:43:00 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:43:01.118709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38338779-9d77-49fb-b031-511f45d3e418 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:01.131768 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38338779-9d77-49fb-b031-511f45d3e418 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:01.136533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4c709-5cd8-44af-a779-3e756cc645f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:01.137206 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4c709-5cd8-44af-a779-3e756cc645f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:01.137490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4c709-5cd8-44af-a779-3e756cc645f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:01.137764 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4c709-5cd8-44af-a779-3e756cc645f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:01.138409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4c709-5cd8-44af-a779-3e756cc645f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:01.139990 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4c709-5cd8-44af-a779-3e756cc645f3 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:01.152244 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1175/2349] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:01.156161 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd308993-ebd6-4fd5-a982-4cfde7c80c4e admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:01.167250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd308993-ebd6-4fd5-a982-4cfde7c80c4e admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:01.172540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917b81d-3df5-4912-91dc-9f89cf431653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:01.173238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917b81d-3df5-4912-91dc-9f89cf431653 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:01.173553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917b81d-3df5-4912-91dc-9f89cf431653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:01.173888 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917b81d-3df5-4912-91dc-9f89cf431653 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:01.174610 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917b81d-3df5-4912-91dc-9f89cf431653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:01.176430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917b81d-3df5-4912-91dc-9f89cf431653 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:01.191772 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1175/2350] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:01.198507 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4c709-5cd8-44af-a779-3e756cc645f3 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:01.209262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4c709-5cd8-44af-a779-3e756cc645f3 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:01.213878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea983bd9-e7e6-4c64-a1d2-c116e691e617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:01.214618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea983bd9-e7e6-4c64-a1d2-c116e691e617 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:01.214966 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea983bd9-e7e6-4c64-a1d2-c116e691e617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:01.215252 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea983bd9-e7e6-4c64-a1d2-c116e691e617 admin admin] PATH_INFO: `/v3/projects/644922c27a004ba6b7eaa246f09850a5/users/e689ae883dda483b80a57291dfd2a3df/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:01.237596 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea983bd9-e7e6-4c64-a1d2-c116e691e617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=644922c27a004ba6b7eaa246f09850a5, user_id=e689ae883dda483b80a57291dfd2a3df, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:01.240696 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea983bd9-e7e6-4c64-a1d2-c116e691e617 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:01.337905 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1176/2351] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:43:01 2022] PUT /identity/v3/projects/644922c27a004ba6b7eaa246f09850a5/users/e689ae883dda483b80a57291dfd2a3df/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:01.357633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8ab63-0b85-44ff-a2d0-87b73395d71f None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:01.357950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8ab63-0b85-44ff-a2d0-87b73395d71f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:01.358639 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8ab63-0b85-44ff-a2d0-87b73395d71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:01.398260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-44d8ab63-0b85-44ff-a2d0-87b73395d71f None None] MFA Rules not processed for user `e689ae883dda483b80a57291dfd2a3df`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:01.399643 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-44d8ab63-0b85-44ff-a2d0-87b73395d71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:01.453491 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:01.477200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1176/2352] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:01.497183 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea983bd9-e7e6-4c64-a1d2-c116e691e617 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:01.585254 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:01.599778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea983bd9-e7e6-4c64-a1d2-c116e691e617 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:01.604890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb82563-b08c-4183-b4c2-b42af9baa074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:01.609411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb82563-b08c-4183-b4c2-b42af9baa074 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:01.614644 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb82563-b08c-4183-b4c2-b42af9baa074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:01.614644 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb82563-b08c-4183-b4c2-b42af9baa074 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:01.614644 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb82563-b08c-4183-b4c2-b42af9baa074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:01.614644 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb82563-b08c-4183-b4c2-b42af9baa074 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:01.765479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1177/2353] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:43:01 2022] POST /identity/v3/projects => generated 387 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:43:01.770127 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8ab63-0b85-44ff-a2d0-87b73395d71f None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:01.781577 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8ab63-0b85-44ff-a2d0-87b73395d71f None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:01.851531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f672bf-16cb-45dc-85b8-75a6f698af8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:01.852158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f672bf-16cb-45dc-85b8-75a6f698af8c admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:01.852440 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f672bf-16cb-45dc-85b8-75a6f698af8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:01.852709 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f672bf-16cb-45dc-85b8-75a6f698af8c admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:01.853398 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f672bf-16cb-45dc-85b8-75a6f698af8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:01.856035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f672bf-16cb-45dc-85b8-75a6f698af8c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:01.982671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1177/2354] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:43:01 2022] POST /identity/v3/users => generated 300 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:43:01.987434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb82563-b08c-4183-b4c2-b42af9baa074 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:01.998518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb82563-b08c-4183-b4c2-b42af9baa074 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:02.003138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104f0c1-918d-4671-bf6c-135ba223ab26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:02.003727 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104f0c1-918d-4671-bf6c-135ba223ab26 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:02.004000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104f0c1-918d-4671-bf6c-135ba223ab26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:02.004270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104f0c1-918d-4671-bf6c-135ba223ab26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:02.004934 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6104f0c1-918d-4671-bf6c-135ba223ab26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:02.007039 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6104f0c1-918d-4671-bf6c-135ba223ab26 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:02.016424 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1178/2355] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:02.020992 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f672bf-16cb-45dc-85b8-75a6f698af8c admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:02.031158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f672bf-16cb-45dc-85b8-75a6f698af8c admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:02.035761 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee83b8-097e-41a6-8326-85b3ee7b9952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:02.036937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee83b8-097e-41a6-8326-85b3ee7b9952 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:02.037261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee83b8-097e-41a6-8326-85b3ee7b9952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:02.037534 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee83b8-097e-41a6-8326-85b3ee7b9952 admin admin] PATH_INFO: `/v3/projects/40163589699a4633ab30cfb596bf2611/users/f58f04b138eb4a36a9ed0e7e66278be2/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:02.055904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee83b8-097e-41a6-8326-85b3ee7b9952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40163589699a4633ab30cfb596bf2611, user_id=f58f04b138eb4a36a9ed0e7e66278be2, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:02.059375 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee83b8-097e-41a6-8326-85b3ee7b9952 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:02.149661 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1178/2356] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:43:02 2022] PUT /identity/v3/projects/40163589699a4633ab30cfb596bf2611/users/f58f04b138eb4a36a9ed0e7e66278be2/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:02.153828 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104f0c1-918d-4671-bf6c-135ba223ab26 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:02.165004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104f0c1-918d-4671-bf6c-135ba223ab26 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:02.227661 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58278a8-5011-4f75-bc00-820edea85c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:02.230883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58278a8-5011-4f75-bc00-820edea85c40 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:02.230883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58278a8-5011-4f75-bc00-820edea85c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:02.230883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58278a8-5011-4f75-bc00-820edea85c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:02.230883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58278a8-5011-4f75-bc00-820edea85c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:02.232086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58278a8-5011-4f75-bc00-820edea85c40 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:02.243728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1179/2357] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:02.247931 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee83b8-097e-41a6-8326-85b3ee7b9952 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:02.258863 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee83b8-097e-41a6-8326-85b3ee7b9952 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:02.263427 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f10b20-a1ed-44b2-984f-cd29270a4b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:02.264070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f10b20-a1ed-44b2-984f-cd29270a4b30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:02.264347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f10b20-a1ed-44b2-984f-cd29270a4b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:02.264620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f10b20-a1ed-44b2-984f-cd29270a4b30 admin admin] PATH_INFO: `/v3/projects/40163589699a4633ab30cfb596bf2611/users/f58f04b138eb4a36a9ed0e7e66278be2/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:02.266625 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f10b20-a1ed-44b2-984f-cd29270a4b30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40163589699a4633ab30cfb596bf2611, user_id=f58f04b138eb4a36a9ed0e7e66278be2, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:02.269295 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f10b20-a1ed-44b2-984f-cd29270a4b30 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:02.280904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1179/2358] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:43:02 2022] PUT /identity/v3/projects/40163589699a4633ab30cfb596bf2611/users/f58f04b138eb4a36a9ed0e7e66278be2/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:02.295179 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542c089-2121-4198-815a-4ad6f7348f1e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:02.295812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542c089-2121-4198-815a-4ad6f7348f1e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:02.296388 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542c089-2121-4198-815a-4ad6f7348f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:02.333830 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1542c089-2121-4198-815a-4ad6f7348f1e None None] MFA Rules not processed for user `f58f04b138eb4a36a9ed0e7e66278be2`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:02.335197 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1542c089-2121-4198-815a-4ad6f7348f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:02.406426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1180/2359] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:02.418030 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f10b20-a1ed-44b2-984f-cd29270a4b30 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:02.419799 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-39f10b20-a1ed-44b2-984f-cd29270a4b30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:02.501343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f10b20-a1ed-44b2-984f-cd29270a4b30 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:02.505696 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011e4b8-3b3a-47f5-83a4-ee3daaa6699b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:02.506212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011e4b8-3b3a-47f5-83a4-ee3daaa6699b service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:02.506487 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011e4b8-3b3a-47f5-83a4-ee3daaa6699b service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:02.511222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011e4b8-3b3a-47f5-83a4-ee3daaa6699b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:02.516605 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011e4b8-3b3a-47f5-83a4-ee3daaa6699b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:02.518252 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011e4b8-3b3a-47f5-83a4-ee3daaa6699b service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:02.529297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1180/2360] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:43:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:43:02.602226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542c089-2121-4198-815a-4ad6f7348f1e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:02.650393 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:02.676899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542c089-2121-4198-815a-4ad6f7348f1e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:02.676899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a1136-a7ab-42c8-b71c-fc28aa9187e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:02.676899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a1136-a7ab-42c8-b71c-fc28aa9187e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:02.676899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a1136-a7ab-42c8-b71c-fc28aa9187e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:02.679010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a1136-a7ab-42c8-b71c-fc28aa9187e3 admin admin] PATH_INFO: `/v3/users/e689ae883dda483b80a57291dfd2a3df` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:02.679010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a1136-a7ab-42c8-b71c-fc28aa9187e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e689ae883dda483b80a57291dfd2a3df)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:02.679010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a1136-a7ab-42c8-b71c-fc28aa9187e3 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:02.995478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-700a1136-a7ab-42c8-b71c-fc28aa9187e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e689ae883dda483b80a57291dfd2a3df'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:02.996421 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-700a1136-a7ab-42c8-b71c-fc28aa9187e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e689ae883dda483b80a57291dfd2a3df'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:03.107529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-700a1136-a7ab-42c8-b71c-fc28aa9187e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e689ae883dda483b80a57291dfd2a3df'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:03.108164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-700a1136-a7ab-42c8-b71c-fc28aa9187e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e689ae883dda483b80a57291dfd2a3df'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:03.133192 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-700a1136-a7ab-42c8-b71c-fc28aa9187e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e689ae883dda483b80a57291dfd2a3df'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:03.148132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1181/2361] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:43:02 2022] DELETE /identity/v3/users/e689ae883dda483b80a57291dfd2a3df => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:43:03.153584 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5551d5-fe37-4719-b1cf-954adad08296 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:03.154549 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5551d5-fe37-4719-b1cf-954adad08296 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:03.155048 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5551d5-fe37-4719-b1cf-954adad08296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:03.179823 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-9d5551d5-fe37-4719-b1cf-954adad08296 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:03.179823 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-9d5551d5-fe37-4719-b1cf-954adad08296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:03.218655 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:03.269464 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1181/2362] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:03.280002 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a1136-a7ab-42c8-b71c-fc28aa9187e3 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:03.281553 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-700a1136-a7ab-42c8-b71c-fc28aa9187e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:03.348269 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a1136-a7ab-42c8-b71c-fc28aa9187e3 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:03.352856 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec04589-3514-4d45-b819-5a095601258a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:03.353815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec04589-3514-4d45-b819-5a095601258a service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:03.354110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec04589-3514-4d45-b819-5a095601258a service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:03.354442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec04589-3514-4d45-b819-5a095601258a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:03.366934 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec04589-3514-4d45-b819-5a095601258a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:03.368429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec04589-3514-4d45-b819-5a095601258a service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:03.378308 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1182/2363] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:43:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:43:03.839761 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db227c26-c99b-47f7-bc5b-f3ec0a24a604 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:03.840041 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db227c26-c99b-47f7-bc5b-f3ec0a24a604 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:03.840312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db227c26-c99b-47f7-bc5b-f3ec0a24a604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:03.872250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-db227c26-c99b-47f7-bc5b-f3ec0a24a604 None None] MFA Rules not processed for user `4ae237601b4a4aab9e969fedb5ee37d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:03.873464 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-db227c26-c99b-47f7-bc5b-f3ec0a24a604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:03.898802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec04589-3514-4d45-b819-5a095601258a service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:03.900467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-fec04589-3514-4d45-b819-5a095601258a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:03.914422 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec04589-3514-4d45-b819-5a095601258a service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:03.919744 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254030a-33f8-4fe3-a235-98cd39912308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:03.920743 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254030a-33f8-4fe3-a235-98cd39912308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:03.921025 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254030a-33f8-4fe3-a235-98cd39912308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:03.921332 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254030a-33f8-4fe3-a235-98cd39912308 admin admin] PATH_INFO: `/v3/projects/644922c27a004ba6b7eaa246f09850a5` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:03.922797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254030a-33f8-4fe3-a235-98cd39912308 admin admin] RBAC: Authorizing `identity:delete_project(project_id=644922c27a004ba6b7eaa246f09850a5)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:03.924706 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254030a-33f8-4fe3-a235-98cd39912308 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:03.939959 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1182/2364] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Sep 19 10:43:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:03.954005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db227c26-c99b-47f7-bc5b-f3ec0a24a604 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:03.956507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-db227c26-c99b-47f7-bc5b-f3ec0a24a604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:04.029203 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db227c26-c99b-47f7-bc5b-f3ec0a24a604 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:04.034681 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff8334-c138-4f65-a036-4d99ad7a919e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17a8bc6c506c45be897bda23eaab485a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:04.035259 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff8334-c138-4f65-a036-4d99ad7a919e service swift] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:04.035538 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff8334-c138-4f65-a036-4d99ad7a919e service swift] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:04.035824 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff8334-c138-4f65-a036-4d99ad7a919e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:04.043944 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff8334-c138-4f65-a036-4d99ad7a919e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:04.053919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff8334-c138-4f65-a036-4d99ad7a919e service swift] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:04.054870 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1183/2365] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 19 10:43:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 10:43:04.075569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b254030a-33f8-4fe3-a235-98cd39912308 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '644922c27a004ba6b7eaa246f09850a5', 'request_id': 'req-b254030a-33f8-4fe3-a235-98cd39912308'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:04.081369 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b254030a-33f8-4fe3-a235-98cd39912308 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '644922c27a004ba6b7eaa246f09850a5', 'request_id': 'req-b254030a-33f8-4fe3-a235-98cd39912308'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:04.087631 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b254030a-33f8-4fe3-a235-98cd39912308 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '644922c27a004ba6b7eaa246f09850a5', 'request_id': 'req-b254030a-33f8-4fe3-a235-98cd39912308'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:04.089309 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:04.205296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b254030a-33f8-4fe3-a235-98cd39912308 admin admin] The token cache is being invalidate because project 644922c27a004ba6b7eaa246f09850a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:43:04.205641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b254030a-33f8-4fe3-a235-98cd39912308 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:04.206516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b254030a-33f8-4fe3-a235-98cd39912308 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:04.209130 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1183/2366] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:43:03 2022] DELETE /identity/v3/projects/644922c27a004ba6b7eaa246f09850a5 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:43:04.212947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff8334-c138-4f65-a036-4d99ad7a919e service swift] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:04.225765 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff8334-c138-4f65-a036-4d99ad7a919e service swift] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:04.294744 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:04.310975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb1615-65b6-430a-b9de-2c4cbd7d665d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:04.311856 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb1615-65b6-430a-b9de-2c4cbd7d665d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:04.312236 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb1615-65b6-430a-b9de-2c4cbd7d665d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:04.312615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb1615-65b6-430a-b9de-2c4cbd7d665d admin admin] PATH_INFO: `/v3/users/f58f04b138eb4a36a9ed0e7e66278be2` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:04.314180 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb1615-65b6-430a-b9de-2c4cbd7d665d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f58f04b138eb4a36a9ed0e7e66278be2)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:04.318082 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb1615-65b6-430a-b9de-2c4cbd7d665d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:04.593431 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-d8eb1615-65b6-430a-b9de-2c4cbd7d665d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f58f04b138eb4a36a9ed0e7e66278be2'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:04.689809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-d8eb1615-65b6-430a-b9de-2c4cbd7d665d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f58f04b138eb4a36a9ed0e7e66278be2'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:04.690910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-d8eb1615-65b6-430a-b9de-2c4cbd7d665d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f58f04b138eb4a36a9ed0e7e66278be2'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:04.692132 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-d8eb1615-65b6-430a-b9de-2c4cbd7d665d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f58f04b138eb4a36a9ed0e7e66278be2'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:04.701601 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-d8eb1615-65b6-430a-b9de-2c4cbd7d665d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f58f04b138eb4a36a9ed0e7e66278be2'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:04.725220 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1184/2367] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:43:04 2022] DELETE /identity/v3/users/f58f04b138eb4a36a9ed0e7e66278be2 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:05.328672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254030a-33f8-4fe3-a235-98cd39912308 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:05.337227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b254030a-33f8-4fe3-a235-98cd39912308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:05.351746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254030a-33f8-4fe3-a235-98cd39912308 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:05.476375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea117440-4c87-42bb-bebb-9c9f385cb172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:05.477125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea117440-4c87-42bb-bebb-9c9f385cb172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:05.477480 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea117440-4c87-42bb-bebb-9c9f385cb172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:05.477912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea117440-4c87-42bb-bebb-9c9f385cb172 admin admin] PATH_INFO: `/v3/projects/40163589699a4633ab30cfb596bf2611` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:05.479474 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea117440-4c87-42bb-bebb-9c9f385cb172 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40163589699a4633ab30cfb596bf2611)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:05.483049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea117440-4c87-42bb-bebb-9c9f385cb172 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:05.629039 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ea117440-4c87-42bb-bebb-9c9f385cb172 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40163589699a4633ab30cfb596bf2611', 'request_id': 'req-ea117440-4c87-42bb-bebb-9c9f385cb172'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:05.636431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ea117440-4c87-42bb-bebb-9c9f385cb172 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40163589699a4633ab30cfb596bf2611', 'request_id': 'req-ea117440-4c87-42bb-bebb-9c9f385cb172'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:05.644441 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ea117440-4c87-42bb-bebb-9c9f385cb172 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40163589699a4633ab30cfb596bf2611', 'request_id': 'req-ea117440-4c87-42bb-bebb-9c9f385cb172'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:05.788710 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ea117440-4c87-42bb-bebb-9c9f385cb172 admin admin] The token cache is being invalidate because project 40163589699a4633ab30cfb596bf2611 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:43:05.789169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ea117440-4c87-42bb-bebb-9c9f385cb172 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:05.789934 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ea117440-4c87-42bb-bebb-9c9f385cb172 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:05.792176 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1184/2368] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:43:05 2022] DELETE /identity/v3/projects/40163589699a4633ab30cfb596bf2611 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:05.798083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb61aa7-165a-48ed-9926-99bfaf40be87 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:05.798661 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb61aa7-165a-48ed-9926-99bfaf40be87 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:05.798843 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb61aa7-165a-48ed-9926-99bfaf40be87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:05.836718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-4bb61aa7-165a-48ed-9926-99bfaf40be87 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:05.838212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4bb61aa7-165a-48ed-9926-99bfaf40be87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:05.962130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1185/2369] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:05.972607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63cfd6-ce7a-47c9-abe9-476b86964d60 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:05.972958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63cfd6-ce7a-47c9-abe9-476b86964d60 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:05.973940 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63cfd6-ce7a-47c9-abe9-476b86964d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:06.005326 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6f63cfd6-ce7a-47c9-abe9-476b86964d60 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:06.006838 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6f63cfd6-ce7a-47c9-abe9-476b86964d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:06.022219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb61aa7-165a-48ed-9926-99bfaf40be87 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:06.029319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1185/2370] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:06.035136 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb61aa7-165a-48ed-9926-99bfaf40be87 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:06.041602 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33925daa-7a44-4091-9ba3-2c0061dc779c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:06.042365 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33925daa-7a44-4091-9ba3-2c0061dc779c admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:06.042701 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33925daa-7a44-4091-9ba3-2c0061dc779c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:06.042983 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33925daa-7a44-4091-9ba3-2c0061dc779c admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:06.048817 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33925daa-7a44-4091-9ba3-2c0061dc779c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:43:06.048817 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33925daa-7a44-4091-9ba3-2c0061dc779c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:06.048817 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33925daa-7a44-4091-9ba3-2c0061dc779c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:06.065750 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63cfd6-ce7a-47c9-abe9-476b86964d60 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:06.073278 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1186/2371] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:43:06.078444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63cfd6-ce7a-47c9-abe9-476b86964d60 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:06.084739 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e4945-1987-4158-a8d9-c9ec81f4a89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:06.085556 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e4945-1987-4158-a8d9-c9ec81f4a89e admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:06.086251 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e4945-1987-4158-a8d9-c9ec81f4a89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:06.086987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e4945-1987-4158-a8d9-c9ec81f4a89e admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:06.088131 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571e4945-1987-4158-a8d9-c9ec81f4a89e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:06.090543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571e4945-1987-4158-a8d9-c9ec81f4a89e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:06.211810 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33925daa-7a44-4091-9ba3-2c0061dc779c admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:06.213169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1186/2372] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:43:06 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:43:06.222902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33925daa-7a44-4091-9ba3-2c0061dc779c admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:06.288188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:06.303996 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74d74f-85d0-43ae-87d5-6c0ff695ce00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:06.305028 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74d74f-85d0-43ae-87d5-6c0ff695ce00 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:06.305625 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74d74f-85d0-43ae-87d5-6c0ff695ce00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:06.305926 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74d74f-85d0-43ae-87d5-6c0ff695ce00 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:06.306540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74d74f-85d0-43ae-87d5-6c0ff695ce00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:06.308666 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74d74f-85d0-43ae-87d5-6c0ff695ce00 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:06.418754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1187/2373] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:43:06 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:43:06.423719 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e4945-1987-4158-a8d9-c9ec81f4a89e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:06.431818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e4945-1987-4158-a8d9-c9ec81f4a89e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:06.436698 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843233a2-9a3b-462d-9fcf-67b899520f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:06.437412 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843233a2-9a3b-462d-9fcf-67b899520f91 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:06.437745 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843233a2-9a3b-462d-9fcf-67b899520f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:06.438071 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843233a2-9a3b-462d-9fcf-67b899520f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:06.438789 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843233a2-9a3b-462d-9fcf-67b899520f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:06.440374 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843233a2-9a3b-462d-9fcf-67b899520f91 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:06.456566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1187/2374] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:43:06.460700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74d74f-85d0-43ae-87d5-6c0ff695ce00 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:06.470979 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74d74f-85d0-43ae-87d5-6c0ff695ce00 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:06.481812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b401c-e502-4886-9cb4-92767c1e359a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:06.482845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b401c-e502-4886-9cb4-92767c1e359a admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:06.482967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b401c-e502-4886-9cb4-92767c1e359a admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:06.483163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b401c-e502-4886-9cb4-92767c1e359a admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:06.483888 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b401c-e502-4886-9cb4-92767c1e359a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:06.485741 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b401c-e502-4886-9cb4-92767c1e359a admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:06.498061 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1188/2375] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:06.504072 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843233a2-9a3b-462d-9fcf-67b899520f91 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:06.515518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843233a2-9a3b-462d-9fcf-67b899520f91 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:06.521349 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b1c75-1f18-4fc4-a94a-61dc1a26951c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:06.522146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b1c75-1f18-4fc4-a94a-61dc1a26951c admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:06.532682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b1c75-1f18-4fc4-a94a-61dc1a26951c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:06.532682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b1c75-1f18-4fc4-a94a-61dc1a26951c admin admin] PATH_INFO: `/v3/projects/6fcbef71a50440adbd7070148cb8f16e/users/aa03870b7b874d29897cd2d947cb7ee1/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:06.542259 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b1c75-1f18-4fc4-a94a-61dc1a26951c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fcbef71a50440adbd7070148cb8f16e, user_id=aa03870b7b874d29897cd2d947cb7ee1, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:06.545832 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b1c75-1f18-4fc4-a94a-61dc1a26951c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:06.677572 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f3571-f072-4682-b618-dd7f03c3c97a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:06.678132 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f3571-f072-4682-b618-dd7f03c3c97a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:06.678409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f3571-f072-4682-b618-dd7f03c3c97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:06.681161 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1188/2376] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:43:06 2022] PUT /identity/v3/projects/6fcbef71a50440adbd7070148cb8f16e/users/aa03870b7b874d29897cd2d947cb7ee1/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:06.738420 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-e20f3571-f072-4682-b618-dd7f03c3c97a None None] MFA Rules not processed for user `aa03870b7b874d29897cd2d947cb7ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:06.740393 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e20f3571-f072-4682-b618-dd7f03c3c97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:06.865450 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1189/2377] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:06.876525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b1c75-1f18-4fc4-a94a-61dc1a26951c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:06.904196 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:06.952373 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b1c75-1f18-4fc4-a94a-61dc1a26951c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:06.960887 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245df293-3f0c-4866-b2b1-919c98505082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:06.961835 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245df293-3f0c-4866-b2b1-919c98505082 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:06.962189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245df293-3f0c-4866-b2b1-919c98505082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:06.962545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245df293-3f0c-4866-b2b1-919c98505082 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:06.963857 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245df293-3f0c-4866-b2b1-919c98505082 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:06.968623 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245df293-3f0c-4866-b2b1-919c98505082 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:07.068794 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1189/2378] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:43:06 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:43:07.073693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f3571-f072-4682-b618-dd7f03c3c97a None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:07.090335 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f3571-f072-4682-b618-dd7f03c3c97a None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:07.190922 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7cd37-8954-41e2-9dbf-e8932af887a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:07.191446 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7cd37-8954-41e2-9dbf-e8932af887a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:07.192422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7cd37-8954-41e2-9dbf-e8932af887a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:07.192760 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7cd37-8954-41e2-9dbf-e8932af887a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:07.193494 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7cd37-8954-41e2-9dbf-e8932af887a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:07.195724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7cd37-8954-41e2-9dbf-e8932af887a4 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:07.340254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1190/2379] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:43:07 2022] POST /identity/v3/users => generated 299 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:43:07.341927 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245df293-3f0c-4866-b2b1-919c98505082 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:07.354682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245df293-3f0c-4866-b2b1-919c98505082 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:07.360142 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49db24-691e-4431-ad93-601f64f8924e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:07.360857 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49db24-691e-4431-ad93-601f64f8924e admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:07.361163 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49db24-691e-4431-ad93-601f64f8924e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:07.361489 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49db24-691e-4431-ad93-601f64f8924e admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:07.363301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d49db24-691e-4431-ad93-601f64f8924e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:07.365955 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d49db24-691e-4431-ad93-601f64f8924e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:07.381930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1190/2380] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:07.389432 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7cd37-8954-41e2-9dbf-e8932af887a4 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:07.403523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7cd37-8954-41e2-9dbf-e8932af887a4 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:07.409482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c08689-401c-46a0-85b6-a3baa571284d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:07.410239 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c08689-401c-46a0-85b6-a3baa571284d admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:07.410636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c08689-401c-46a0-85b6-a3baa571284d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:07.410922 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c08689-401c-46a0-85b6-a3baa571284d admin admin] PATH_INFO: `/v3/projects/02704351883a486b87b4fab4deb33f4a/users/8be4a2840e3f4a2093b2c8922ab3404f/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:07.429833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c08689-401c-46a0-85b6-a3baa571284d admin admin] RBAC: Authorizing `identity:create_grant(project_id=02704351883a486b87b4fab4deb33f4a, user_id=8be4a2840e3f4a2093b2c8922ab3404f, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:07.433902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c08689-401c-46a0-85b6-a3baa571284d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:07.519254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1191/2381] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:43:07 2022] PUT /identity/v3/projects/02704351883a486b87b4fab4deb33f4a/users/8be4a2840e3f4a2093b2c8922ab3404f/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:07.524338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49db24-691e-4431-ad93-601f64f8924e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:07.534681 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49db24-691e-4431-ad93-601f64f8924e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:07.597222 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309717ad-25ee-4fb6-872e-51017622af86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:07.597834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309717ad-25ee-4fb6-872e-51017622af86 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:07.598113 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309717ad-25ee-4fb6-872e-51017622af86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:07.598442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309717ad-25ee-4fb6-872e-51017622af86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:07.599082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309717ad-25ee-4fb6-872e-51017622af86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:07.600797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309717ad-25ee-4fb6-872e-51017622af86 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:07.610842 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1191/2382] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:07.616117 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c08689-401c-46a0-85b6-a3baa571284d admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:07.628198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c08689-401c-46a0-85b6-a3baa571284d admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:07.633757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a960ac-8889-4e54-b973-258eeb210306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:07.634562 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a960ac-8889-4e54-b973-258eeb210306 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:07.634900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a960ac-8889-4e54-b973-258eeb210306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:07.635181 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a960ac-8889-4e54-b973-258eeb210306 admin admin] PATH_INFO: `/v3/projects/02704351883a486b87b4fab4deb33f4a/users/8be4a2840e3f4a2093b2c8922ab3404f/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:07.637572 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a960ac-8889-4e54-b973-258eeb210306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02704351883a486b87b4fab4deb33f4a, user_id=8be4a2840e3f4a2093b2c8922ab3404f, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:07.640938 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a960ac-8889-4e54-b973-258eeb210306 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:07.655995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1192/2383] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:43:07 2022] PUT /identity/v3/projects/02704351883a486b87b4fab4deb33f4a/users/8be4a2840e3f4a2093b2c8922ab3404f/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:07.666876 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34425c-11fd-4297-8d16-bfb0cc1af808 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:07.667291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34425c-11fd-4297-8d16-bfb0cc1af808 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:07.667830 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34425c-11fd-4297-8d16-bfb0cc1af808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:07.701298 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1c34425c-11fd-4297-8d16-bfb0cc1af808 None None] MFA Rules not processed for user `8be4a2840e3f4a2093b2c8922ab3404f`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:07.702532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1c34425c-11fd-4297-8d16-bfb0cc1af808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:07.810814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1192/2384] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:07.830582 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a960ac-8889-4e54-b973-258eeb210306 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:07.832703 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b5a960ac-8889-4e54-b973-258eeb210306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:07.854990 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:07.912958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a960ac-8889-4e54-b973-258eeb210306 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:07.917340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe6ce0-318a-4e58-a7cc-88b6141e5fa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:07.918045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe6ce0-318a-4e58-a7cc-88b6141e5fa0 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:07.918371 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe6ce0-318a-4e58-a7cc-88b6141e5fa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:07.918677 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe6ce0-318a-4e58-a7cc-88b6141e5fa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:07.932966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe6ce0-318a-4e58-a7cc-88b6141e5fa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:07.934654 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe6ce0-318a-4e58-a7cc-88b6141e5fa0 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:07.948389 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1193/2385] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:43:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:43:09.272079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34425c-11fd-4297-8d16-bfb0cc1af808 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:09.289602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:09.349342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34425c-11fd-4297-8d16-bfb0cc1af808 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:09.355484 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea7aa7-42d1-4147-abf9-06eb906699eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:09.356387 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea7aa7-42d1-4147-abf9-06eb906699eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:09.356698 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea7aa7-42d1-4147-abf9-06eb906699eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:09.357101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea7aa7-42d1-4147-abf9-06eb906699eb admin admin] PATH_INFO: `/v3/users/aa03870b7b874d29897cd2d947cb7ee1` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:09.358628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea7aa7-42d1-4147-abf9-06eb906699eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa03870b7b874d29897cd2d947cb7ee1)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:09.360997 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea7aa7-42d1-4147-abf9-06eb906699eb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:09.724164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ceea7aa7-42d1-4147-abf9-06eb906699eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa03870b7b874d29897cd2d947cb7ee1'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:09.736729 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ceea7aa7-42d1-4147-abf9-06eb906699eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa03870b7b874d29897cd2d947cb7ee1'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:09.813235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ceea7aa7-42d1-4147-abf9-06eb906699eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa03870b7b874d29897cd2d947cb7ee1'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:09.813627 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ceea7aa7-42d1-4147-abf9-06eb906699eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa03870b7b874d29897cd2d947cb7ee1'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:09.829688 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ceea7aa7-42d1-4147-abf9-06eb906699eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa03870b7b874d29897cd2d947cb7ee1'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:09.845438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1193/2386] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:43:09 2022] DELETE /identity/v3/users/aa03870b7b874d29897cd2d947cb7ee1 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:09.850387 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e07e2-da5e-493b-9c76-eef39b07d0a4 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:09.850854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e07e2-da5e-493b-9c76-eef39b07d0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:09.850950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e07e2-da5e-493b-9c76-eef39b07d0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:09.873880 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-520e07e2-da5e-493b-9c76-eef39b07d0a4 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:09.873880 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-520e07e2-da5e-493b-9c76-eef39b07d0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:09.934647 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1194/2387] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:09.945921 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea7aa7-42d1-4147-abf9-06eb906699eb admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:09.947371 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ceea7aa7-42d1-4147-abf9-06eb906699eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:10.018618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea7aa7-42d1-4147-abf9-06eb906699eb admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:10.059280 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354d211-00ca-4dc4-8f24-2b1c98347d49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:10.059280 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354d211-00ca-4dc4-8f24-2b1c98347d49 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:10.059280 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354d211-00ca-4dc4-8f24-2b1c98347d49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:10.059280 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354d211-00ca-4dc4-8f24-2b1c98347d49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:10.060168 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354d211-00ca-4dc4-8f24-2b1c98347d49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:10.060168 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354d211-00ca-4dc4-8f24-2b1c98347d49 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:10.060168 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1194/2388] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:43:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:43:10.562771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e07e2-da5e-493b-9c76-eef39b07d0a4 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:10.567327 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-520e07e2-da5e-493b-9c76-eef39b07d0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:10.581742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e07e2-da5e-493b-9c76-eef39b07d0a4 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:10.587378 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7a75e-44ed-421e-9613-789c45612f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:10.588092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7a75e-44ed-421e-9613-789c45612f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:10.588390 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7a75e-44ed-421e-9613-789c45612f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:10.588664 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7a75e-44ed-421e-9613-789c45612f0b admin admin] PATH_INFO: `/v3/projects/6fcbef71a50440adbd7070148cb8f16e` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:10.589803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7a75e-44ed-421e-9613-789c45612f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fcbef71a50440adbd7070148cb8f16e)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:10.591790 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7a75e-44ed-421e-9613-789c45612f0b admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:10.747435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-48b7a75e-44ed-421e-9613-789c45612f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fcbef71a50440adbd7070148cb8f16e', 'request_id': 'req-48b7a75e-44ed-421e-9613-789c45612f0b'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:10.753904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-48b7a75e-44ed-421e-9613-789c45612f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fcbef71a50440adbd7070148cb8f16e', 'request_id': 'req-48b7a75e-44ed-421e-9613-789c45612f0b'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:10.761345 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-48b7a75e-44ed-421e-9613-789c45612f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fcbef71a50440adbd7070148cb8f16e', 'request_id': 'req-48b7a75e-44ed-421e-9613-789c45612f0b'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:10.922475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-48b7a75e-44ed-421e-9613-789c45612f0b admin admin] The token cache is being invalidate because project 6fcbef71a50440adbd7070148cb8f16e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:43:10.923042 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-48b7a75e-44ed-421e-9613-789c45612f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:10.923380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-48b7a75e-44ed-421e-9613-789c45612f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:10.924861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1195/2389] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:43:10 2022] DELETE /identity/v3/projects/6fcbef71a50440adbd7070148cb8f16e => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:10.929641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354d211-00ca-4dc4-8f24-2b1c98347d49 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:10.944816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354d211-00ca-4dc4-8f24-2b1c98347d49 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:10.980268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:11.007709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835c550-9bb1-4014-a485-501580741366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:11.008262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835c550-9bb1-4014-a485-501580741366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:11.008537 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835c550-9bb1-4014-a485-501580741366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:11.008832 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835c550-9bb1-4014-a485-501580741366 admin admin] PATH_INFO: `/v3/users/8be4a2840e3f4a2093b2c8922ab3404f` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:11.010017 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835c550-9bb1-4014-a485-501580741366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8be4a2840e3f4a2093b2c8922ab3404f)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:11.017558 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835c550-9bb1-4014-a485-501580741366 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:11.243429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3835c550-9bb1-4014-a485-501580741366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8be4a2840e3f4a2093b2c8922ab3404f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:11.244368 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3835c550-9bb1-4014-a485-501580741366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8be4a2840e3f4a2093b2c8922ab3404f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:11.317835 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3835c550-9bb1-4014-a485-501580741366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8be4a2840e3f4a2093b2c8922ab3404f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:11.326682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3835c550-9bb1-4014-a485-501580741366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8be4a2840e3f4a2093b2c8922ab3404f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:11.331961 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3835c550-9bb1-4014-a485-501580741366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8be4a2840e3f4a2093b2c8922ab3404f'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:11.345561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1195/2390] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:43:10 2022] DELETE /identity/v3/users/8be4a2840e3f4a2093b2c8922ab3404f => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:11.919963 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7a75e-44ed-421e-9613-789c45612f0b admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:11.927323 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-48b7a75e-44ed-421e-9613-789c45612f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:11.958359 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7a75e-44ed-421e-9613-789c45612f0b admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:12.028385 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:12.053141 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72cdb6-e244-4d77-b6c3-2b62e8e018a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:12.053991 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72cdb6-e244-4d77-b6c3-2b62e8e018a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:12.055553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72cdb6-e244-4d77-b6c3-2b62e8e018a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:12.055553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72cdb6-e244-4d77-b6c3-2b62e8e018a7 admin admin] PATH_INFO: `/v3/projects/02704351883a486b87b4fab4deb33f4a` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:12.056385 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72cdb6-e244-4d77-b6c3-2b62e8e018a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02704351883a486b87b4fab4deb33f4a)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:12.058241 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72cdb6-e244-4d77-b6c3-2b62e8e018a7 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:12.213624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2a72cdb6-e244-4d77-b6c3-2b62e8e018a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02704351883a486b87b4fab4deb33f4a', 'request_id': 'req-2a72cdb6-e244-4d77-b6c3-2b62e8e018a7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:12.220166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2a72cdb6-e244-4d77-b6c3-2b62e8e018a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02704351883a486b87b4fab4deb33f4a', 'request_id': 'req-2a72cdb6-e244-4d77-b6c3-2b62e8e018a7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:12.232827 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2a72cdb6-e244-4d77-b6c3-2b62e8e018a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02704351883a486b87b4fab4deb33f4a', 'request_id': 'req-2a72cdb6-e244-4d77-b6c3-2b62e8e018a7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:12.347803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2a72cdb6-e244-4d77-b6c3-2b62e8e018a7 admin admin] The token cache is being invalidate because project 02704351883a486b87b4fab4deb33f4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:43:12.357464 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2a72cdb6-e244-4d77-b6c3-2b62e8e018a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:12.357464 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2a72cdb6-e244-4d77-b6c3-2b62e8e018a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:12.357464 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1196/2391] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:43:11 2022] DELETE /identity/v3/projects/02704351883a486b87b4fab4deb33f4a => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:12.357821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a14710-e845-415e-b342-357a659cb53a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:12.357821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a14710-e845-415e-b342-357a659cb53a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:12.357821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a14710-e845-415e-b342-357a659cb53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:12.377048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-14a14710-e845-415e-b342-357a659cb53a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:12.378409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-14a14710-e845-415e-b342-357a659cb53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:12.448474 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1196/2392] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:43:12.489525 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f6dd2-f9a8-49b7-bf15-a24c91ff8230 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:12.489842 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f6dd2-f9a8-49b7-bf15-a24c91ff8230 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:12.490347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f6dd2-f9a8-49b7-bf15-a24c91ff8230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:12.527813 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-c51f6dd2-f9a8-49b7-bf15-a24c91ff8230 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:12.531829 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c51f6dd2-f9a8-49b7-bf15-a24c91ff8230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:12.540349 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1197/2393] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:12.545114 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a14710-e845-415e-b342-357a659cb53a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:12.557021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a14710-e845-415e-b342-357a659cb53a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:12.561589 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2dd182-8dd3-405d-bcf7-02bfd1bc024d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:12.562486 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dd182-8dd3-405d-bcf7-02bfd1bc024d admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:12.562971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dd182-8dd3-405d-bcf7-02bfd1bc024d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:12.563364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dd182-8dd3-405d-bcf7-02bfd1bc024d admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:12.564162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2dd182-8dd3-405d-bcf7-02bfd1bc024d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:43:12.564581 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2dd182-8dd3-405d-bcf7-02bfd1bc024d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:12.566574 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2dd182-8dd3-405d-bcf7-02bfd1bc024d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:12.580861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1197/2394] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:43:12.587937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f6dd2-f9a8-49b7-bf15-a24c91ff8230 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:12.598416 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f6dd2-f9a8-49b7-bf15-a24c91ff8230 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:12.604992 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb876d-8121-458d-8d3d-bf1a5dba6c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:12.605690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb876d-8121-458d-8d3d-bf1a5dba6c2c admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:12.606011 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb876d-8121-458d-8d3d-bf1a5dba6c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:12.606333 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb876d-8121-458d-8d3d-bf1a5dba6c2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:12.607035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb876d-8121-458d-8d3d-bf1a5dba6c2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:12.609222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb876d-8121-458d-8d3d-bf1a5dba6c2c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:13.904631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1198/2395] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:43:12 2022] POST /identity/v3/projects => generated 385 bytes in 1317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:43:13.909189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2dd182-8dd3-405d-bcf7-02bfd1bc024d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:13.922373 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2dd182-8dd3-405d-bcf7-02bfd1bc024d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:13.974601 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:13.990581 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecb4a3-d2b0-494b-82fb-ff8baadef20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:13.991337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecb4a3-d2b0-494b-82fb-ff8baadef20a admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:13.991615 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecb4a3-d2b0-494b-82fb-ff8baadef20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:13.991885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecb4a3-d2b0-494b-82fb-ff8baadef20a admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:13.992527 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cecb4a3-d2b0-494b-82fb-ff8baadef20a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:13.994466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cecb4a3-d2b0-494b-82fb-ff8baadef20a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:14.112697 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1198/2396] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:43:13 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:43:14.115906 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb876d-8121-458d-8d3d-bf1a5dba6c2c admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:14.126626 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb876d-8121-458d-8d3d-bf1a5dba6c2c admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:14.132472 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630c9e1-da80-4d3a-b00b-f09244360eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:14.133287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630c9e1-da80-4d3a-b00b-f09244360eec admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:14.133596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630c9e1-da80-4d3a-b00b-f09244360eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:14.134219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630c9e1-da80-4d3a-b00b-f09244360eec admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:14.135044 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630c9e1-da80-4d3a-b00b-f09244360eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:14.136996 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630c9e1-da80-4d3a-b00b-f09244360eec admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:14.152416 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1199/2397] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:14.162568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecb4a3-d2b0-494b-82fb-ff8baadef20a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:14.179327 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecb4a3-d2b0-494b-82fb-ff8baadef20a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:14.186564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d6fc6-f643-48d4-b496-d4d479cdd530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:14.193242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d6fc6-f643-48d4-b496-d4d479cdd530 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:14.193242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d6fc6-f643-48d4-b496-d4d479cdd530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:14.193242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d6fc6-f643-48d4-b496-d4d479cdd530 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:14.193242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d6fc6-f643-48d4-b496-d4d479cdd530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:14.193242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d6fc6-f643-48d4-b496-d4d479cdd530 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:14.205377 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1199/2398] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:14.210355 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630c9e1-da80-4d3a-b00b-f09244360eec admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:14.224068 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630c9e1-da80-4d3a-b00b-f09244360eec admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:14.232770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5e0ce-f141-41af-8257-5fe47fcdcaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:14.233709 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5e0ce-f141-41af-8257-5fe47fcdcaef admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:14.234251 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5e0ce-f141-41af-8257-5fe47fcdcaef admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:14.234720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5e0ce-f141-41af-8257-5fe47fcdcaef admin admin] PATH_INFO: `/v3/projects/5f1a61ca4ecd4ea692b179f2de2d9694/users/29bcaf5bdc814d50ad48674876b9cf68/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:14.254368 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d6fc6-f643-48d4-b496-d4d479cdd530 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:14.256918 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cd0d6fc6-f643-48d4-b496-d4d479cdd530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:14.274211 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5e0ce-f141-41af-8257-5fe47fcdcaef admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f1a61ca4ecd4ea692b179f2de2d9694, user_id=29bcaf5bdc814d50ad48674876b9cf68, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:14.277381 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5e0ce-f141-41af-8257-5fe47fcdcaef admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:14.375513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d6fc6-f643-48d4-b496-d4d479cdd530 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:14.382258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973dc59c-bf42-43b5-9ab7-41b9901448e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0ef47a60f54ca1853fd267e308c58b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:14.383021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dc59c-bf42-43b5-9ab7-41b9901448e0 service placement] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:14.383324 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dc59c-bf42-43b5-9ab7-41b9901448e0 service placement] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:14.383633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dc59c-bf42-43b5-9ab7-41b9901448e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:14.385917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-973dc59c-bf42-43b5-9ab7-41b9901448e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:14.399846 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973dc59c-bf42-43b5-9ab7-41b9901448e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:14.401957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973dc59c-bf42-43b5-9ab7-41b9901448e0 service placement] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:14.413783 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1200/2399] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:43:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:43:14.437713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1200/2400] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:43:14 2022] PUT /identity/v3/projects/5f1a61ca4ecd4ea692b179f2de2d9694/users/29bcaf5bdc814d50ad48674876b9cf68/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:14.448809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc14d4f-71ad-488b-a39f-82f861e95ffb None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:14.449124 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc14d4f-71ad-488b-a39f-82f861e95ffb None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:14.449441 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc14d4f-71ad-488b-a39f-82f861e95ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:14.477687 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-cdc14d4f-71ad-488b-a39f-82f861e95ffb None None] MFA Rules not processed for user `29bcaf5bdc814d50ad48674876b9cf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:14.478973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cdc14d4f-71ad-488b-a39f-82f861e95ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:14.504047 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5e0ce-f141-41af-8257-5fe47fcdcaef admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:14.506935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-caa5e0ce-f141-41af-8257-5fe47fcdcaef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:14.557223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1201/2401] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:43:14.560705 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc14d4f-71ad-488b-a39f-82f861e95ffb None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:14.596529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5e0ce-f141-41af-8257-5fe47fcdcaef admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:14.601634 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365efb3-747e-4bee-96d3-3171563ffd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:14.602271 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365efb3-747e-4bee-96d3-3171563ffd89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:14.602544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365efb3-747e-4bee-96d3-3171563ffd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:14.602816 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365efb3-747e-4bee-96d3-3171563ffd89 admin admin] PATH_INFO: `/v3/users/884620036bb44462964db97238091c19` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:14.604090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365efb3-747e-4bee-96d3-3171563ffd89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=884620036bb44462964db97238091c19)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:14.606095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365efb3-747e-4bee-96d3-3171563ffd89 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:14.627756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:14.638477 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc14d4f-71ad-488b-a39f-82f861e95ffb None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:14.644381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54defa1c-9f55-4028-9f02-4828e2480dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:14.645929 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54defa1c-9f55-4028-9f02-4828e2480dbe admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:14.646286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54defa1c-9f55-4028-9f02-4828e2480dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:14.646629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54defa1c-9f55-4028-9f02-4828e2480dbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:14.647519 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54defa1c-9f55-4028-9f02-4828e2480dbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:14.649874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54defa1c-9f55-4028-9f02-4828e2480dbe admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:14.788341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1202/2402] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:43:14 2022] POST /identity/v3/projects => generated 383 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:43:14.792405 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54defa1c-9f55-4028-9f02-4828e2480dbe admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:14.836958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2365efb3-747e-4bee-96d3-3171563ffd89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '884620036bb44462964db97238091c19'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:14.860125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54defa1c-9f55-4028-9f02-4828e2480dbe admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:14.865496 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05555f0e-5e58-44ab-9487-9d35510b3adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:14.866180 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05555f0e-5e58-44ab-9487-9d35510b3adb admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:14.866482 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05555f0e-5e58-44ab-9487-9d35510b3adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:14.866799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05555f0e-5e58-44ab-9487-9d35510b3adb admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:14.867488 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05555f0e-5e58-44ab-9487-9d35510b3adb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:14.869645 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05555f0e-5e58-44ab-9487-9d35510b3adb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:14.894200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2365efb3-747e-4bee-96d3-3171563ffd89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '884620036bb44462964db97238091c19'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:14.901912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2365efb3-747e-4bee-96d3-3171563ffd89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '884620036bb44462964db97238091c19'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:14.901912 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2365efb3-747e-4bee-96d3-3171563ffd89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '884620036bb44462964db97238091c19'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:14.904162 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2365efb3-747e-4bee-96d3-3171563ffd89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '884620036bb44462964db97238091c19'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:14.921764 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1201/2403] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:43:14 2022] DELETE /identity/v3/users/884620036bb44462964db97238091c19 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:14.997756 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1203/2404] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:43:14 2022] POST /identity/v3/users => generated 292 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:43:15.009875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365efb3-747e-4bee-96d3-3171563ffd89 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:15.012971 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-2365efb3-747e-4bee-96d3-3171563ffd89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:15.087383 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365efb3-747e-4bee-96d3-3171563ffd89 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:15.092995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e404b-7e4b-48cf-834a-4d5c86f4f4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:15.093748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e404b-7e4b-48cf-834a-4d5c86f4f4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:15.094026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e404b-7e4b-48cf-834a-4d5c86f4f4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:15.094302 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e404b-7e4b-48cf-834a-4d5c86f4f4a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:15.094980 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e404b-7e4b-48cf-834a-4d5c86f4f4a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:15.096599 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e404b-7e4b-48cf-834a-4d5c86f4f4a7 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:15.107102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1202/2405] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:15.111096 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05555f0e-5e58-44ab-9487-9d35510b3adb admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:15.123251 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05555f0e-5e58-44ab-9487-9d35510b3adb admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:15.128103 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec306f-76e3-43bd-8fe3-fd44a7df3cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:15.128714 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec306f-76e3-43bd-8fe3-fd44a7df3cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:15.128990 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec306f-76e3-43bd-8fe3-fd44a7df3cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:15.129336 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec306f-76e3-43bd-8fe3-fd44a7df3cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:15.130576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec306f-76e3-43bd-8fe3-fd44a7df3cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:15.132169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec306f-76e3-43bd-8fe3-fd44a7df3cf8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:15.142174 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1204/2406] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:15.146574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e404b-7e4b-48cf-834a-4d5c86f4f4a7 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:15.162057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e404b-7e4b-48cf-834a-4d5c86f4f4a7 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:15.167395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707bd8a-0735-4bc2-b99b-42109a3bcce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:15.167597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707bd8a-0735-4bc2-b99b-42109a3bcce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:15.167843 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707bd8a-0735-4bc2-b99b-42109a3bcce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:15.168117 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707bd8a-0735-4bc2-b99b-42109a3bcce8 admin admin] PATH_INFO: `/v3/projects/78fe22a3084d4efbbd0e65f12ee7d61f/users/8c6a23756d584438880a754507775f44/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:15.184121 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707bd8a-0735-4bc2-b99b-42109a3bcce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78fe22a3084d4efbbd0e65f12ee7d61f, user_id=8c6a23756d584438880a754507775f44, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:15.186883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707bd8a-0735-4bc2-b99b-42109a3bcce8 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:15.281161 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec306f-76e3-43bd-8fe3-fd44a7df3cf8 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:15.282884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9bec306f-76e3-43bd-8fe3-fd44a7df3cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:15.297440 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1203/2407] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:43:15 2022] PUT /identity/v3/projects/78fe22a3084d4efbbd0e65f12ee7d61f/users/8c6a23756d584438880a754507775f44/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:43:15.298205 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec306f-76e3-43bd-8fe3-fd44a7df3cf8 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:15.309401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6185c4-9663-416a-9442-38960d8ef77f None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:15.309684 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6185c4-9663-416a-9442-38960d8ef77f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:15.310040 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6185c4-9663-416a-9442-38960d8ef77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:15.341276 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-df6185c4-9663-416a-9442-38960d8ef77f None None] MFA Rules not processed for user `8c6a23756d584438880a754507775f44`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:15.342433 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-df6185c4-9663-416a-9442-38960d8ef77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:15.369866 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39791b-055e-4397-9016-59557fb23c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:15.370566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39791b-055e-4397-9016-59557fb23c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:15.370945 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39791b-055e-4397-9016-59557fb23c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:15.371229 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39791b-055e-4397-9016-59557fb23c6c admin admin] PATH_INFO: `/v3/projects/b082859e089143dda3eefee066e4916d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:15.372378 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39791b-055e-4397-9016-59557fb23c6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b082859e089143dda3eefee066e4916d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:15.374479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39791b-055e-4397-9016-59557fb23c6c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:15.409190 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1204/2408] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:43:15.419119 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6185c4-9663-416a-9442-38960d8ef77f None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:15.429721 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6185c4-9663-416a-9442-38960d8ef77f None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:15.434441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc0edb-afc7-4db9-8306-8ced1e323862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:15.435092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc0edb-afc7-4db9-8306-8ced1e323862 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:15.435294 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc0edb-afc7-4db9-8306-8ced1e323862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:15.435588 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc0edb-afc7-4db9-8306-8ced1e323862 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:15.436227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc0edb-afc7-4db9-8306-8ced1e323862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:15.438242 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc0edb-afc7-4db9-8306-8ced1e323862 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:15.511289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-db39791b-055e-4397-9016-59557fb23c6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b082859e089143dda3eefee066e4916d', 'request_id': 'req-db39791b-055e-4397-9016-59557fb23c6c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:15.517118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-db39791b-055e-4397-9016-59557fb23c6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b082859e089143dda3eefee066e4916d', 'request_id': 'req-db39791b-055e-4397-9016-59557fb23c6c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:15.523891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-db39791b-055e-4397-9016-59557fb23c6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b082859e089143dda3eefee066e4916d', 'request_id': 'req-db39791b-055e-4397-9016-59557fb23c6c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:15.603786 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1205/2409] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:43:15 2022] POST /identity/v3/projects => generated 383 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:43:15.605113 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc0edb-afc7-4db9-8306-8ced1e323862 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:15.639023 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-db39791b-055e-4397-9016-59557fb23c6c admin admin] The token cache is being invalidate because project b082859e089143dda3eefee066e4916d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:43:15.646924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-db39791b-055e-4397-9016-59557fb23c6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:15.646924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-db39791b-055e-4397-9016-59557fb23c6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:15.648320 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1205/2410] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:43:15 2022] DELETE /identity/v3/projects/b082859e089143dda3eefee066e4916d => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:43:15.696681 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc0edb-afc7-4db9-8306-8ced1e323862 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:15.697759 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5f2a5-00ca-4a0b-9ac2-4327748cce60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:15.698501 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5f2a5-00ca-4a0b-9ac2-4327748cce60 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:15.698826 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5f2a5-00ca-4a0b-9ac2-4327748cce60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:15.699177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5f2a5-00ca-4a0b-9ac2-4327748cce60 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:15.700114 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5f2a5-00ca-4a0b-9ac2-4327748cce60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:15.702584 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5f2a5-00ca-4a0b-9ac2-4327748cce60 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:15.822174 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:15.833849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1206/2411] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:43:15 2022] POST /identity/v3/users => generated 298 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:43:15.835464 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39791b-055e-4397-9016-59557fb23c6c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:15.837130 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-db39791b-055e-4397-9016-59557fb23c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:15.909417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39791b-055e-4397-9016-59557fb23c6c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:15.915416 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae713a3-1205-4061-8f4d-87f61493e62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:15.916144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae713a3-1205-4061-8f4d-87f61493e62b admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:15.916507 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae713a3-1205-4061-8f4d-87f61493e62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:15.916891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae713a3-1205-4061-8f4d-87f61493e62b admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:15.917698 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae713a3-1205-4061-8f4d-87f61493e62b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:15.919776 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae713a3-1205-4061-8f4d-87f61493e62b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:15.931113 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1206/2412] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:15.935571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5f2a5-00ca-4a0b-9ac2-4327748cce60 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:15.946054 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5f2a5-00ca-4a0b-9ac2-4327748cce60 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:15.950610 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0defee-3337-41f4-a68c-9e0172e48dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:15.951261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0defee-3337-41f4-a68c-9e0172e48dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:15.951537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0defee-3337-41f4-a68c-9e0172e48dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:15.951852 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0defee-3337-41f4-a68c-9e0172e48dbe admin admin] PATH_INFO: `/v3/projects/98471785d59749aa9e676aa804572df4/users/d7d608cc8b32428aa13edd09caa3c82c/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:15.967395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0defee-3337-41f4-a68c-9e0172e48dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=98471785d59749aa9e676aa804572df4, user_id=d7d608cc8b32428aa13edd09caa3c82c, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:15.970126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0defee-3337-41f4-a68c-9e0172e48dbe admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:16.044291 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1207/2413] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:43:15 2022] PUT /identity/v3/projects/98471785d59749aa9e676aa804572df4/users/d7d608cc8b32428aa13edd09caa3c82c/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:16.048416 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae713a3-1205-4061-8f4d-87f61493e62b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:16.058405 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae713a3-1205-4061-8f4d-87f61493e62b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:16.113627 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d08f5b-9713-46f0-ad9a-453147043f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:16.114269 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d08f5b-9713-46f0-ad9a-453147043f55 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:16.114546 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d08f5b-9713-46f0-ad9a-453147043f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:16.114820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d08f5b-9713-46f0-ad9a-453147043f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:16.115455 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d08f5b-9713-46f0-ad9a-453147043f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:16.117143 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d08f5b-9713-46f0-ad9a-453147043f55 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:16.126484 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1207/2414] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:16.130797 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0defee-3337-41f4-a68c-9e0172e48dbe admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:16.142489 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0defee-3337-41f4-a68c-9e0172e48dbe admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:16.148649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8db0f-0a9c-4bb6-ad0f-21a797c5df15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:16.149471 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8db0f-0a9c-4bb6-ad0f-21a797c5df15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:16.149819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8db0f-0a9c-4bb6-ad0f-21a797c5df15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:16.150128 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8db0f-0a9c-4bb6-ad0f-21a797c5df15 admin admin] PATH_INFO: `/v3/projects/98471785d59749aa9e676aa804572df4/users/d7d608cc8b32428aa13edd09caa3c82c/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:16.152433 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8db0f-0a9c-4bb6-ad0f-21a797c5df15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98471785d59749aa9e676aa804572df4, user_id=d7d608cc8b32428aa13edd09caa3c82c, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:16.155570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8db0f-0a9c-4bb6-ad0f-21a797c5df15 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:16.168655 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1208/2415] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:43:16 2022] PUT /identity/v3/projects/98471785d59749aa9e676aa804572df4/users/d7d608cc8b32428aa13edd09caa3c82c/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:16.181731 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252d91b-e1fc-4406-82a0-000588d234c4 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:16.182018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252d91b-e1fc-4406-82a0-000588d234c4 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:16.182303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252d91b-e1fc-4406-82a0-000588d234c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:16.211930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8252d91b-e1fc-4406-82a0-000588d234c4 None None] MFA Rules not processed for user `d7d608cc8b32428aa13edd09caa3c82c`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:16.213506 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8252d91b-e1fc-4406-82a0-000588d234c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:16.216273 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:16.287343 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1208/2416] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:16.299300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8db0f-0a9c-4bb6-ad0f-21a797c5df15 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:16.301156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-02e8db0f-0a9c-4bb6-ad0f-21a797c5df15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:16.330499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:16.358883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8db0f-0a9c-4bb6-ad0f-21a797c5df15 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:16.362568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81839a45-87a2-4cc5-a3b2-2c7ef042094d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:16.363169 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81839a45-87a2-4cc5-a3b2-2c7ef042094d service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:16.363450 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81839a45-87a2-4cc5-a3b2-2c7ef042094d service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:16.363724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81839a45-87a2-4cc5-a3b2-2c7ef042094d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:16.365601 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-81839a45-87a2-4cc5-a3b2-2c7ef042094d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:16.423968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81839a45-87a2-4cc5-a3b2-2c7ef042094d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:16.425381 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81839a45-87a2-4cc5-a3b2-2c7ef042094d service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:16.434357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1209/2417] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:43:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:43:16.547042 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:16.581296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:18.458887 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252d91b-e1fc-4406-82a0-000588d234c4 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:18.468477 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252d91b-e1fc-4406-82a0-000588d234c4 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:18.472043 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c93932-7f12-4597-b2f5-38749e0c86c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:18.472643 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c93932-7f12-4597-b2f5-38749e0c86c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:18.472923 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c93932-7f12-4597-b2f5-38749e0c86c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:18.474114 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c93932-7f12-4597-b2f5-38749e0c86c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:18.475531 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f1c93932-7f12-4597-b2f5-38749e0c86c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:18.540699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c93932-7f12-4597-b2f5-38749e0c86c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:18.542183 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c93932-7f12-4597-b2f5-38749e0c86c9 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:18.551733 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1209/2418] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:43:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:43:20.215643 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81839a45-87a2-4cc5-a3b2-2c7ef042094d service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:20.226493 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81839a45-87a2-4cc5-a3b2-2c7ef042094d service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:20.230881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bb971-aa30-46fa-aa20-738c49cd888c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:20.231510 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bb971-aa30-46fa-aa20-738c49cd888c service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:20.231798 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bb971-aa30-46fa-aa20-738c49cd888c service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:20.232140 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bb971-aa30-46fa-aa20-738c49cd888c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:20.241613 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907bb971-aa30-46fa-aa20-738c49cd888c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:20.243081 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907bb971-aa30-46fa-aa20-738c49cd888c service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:20.252658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1210/2419] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:43:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:43:21.753291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56d667-91a1-457f-996c-c780e13c3fdb None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:21.753789 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56d667-91a1-457f-996c-c780e13c3fdb None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:21.753895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56d667-91a1-457f-996c-c780e13c3fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:21.775792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-0a56d667-91a1-457f-996c-c780e13c3fdb None None] MFA Rules not processed for user `4ae237601b4a4aab9e969fedb5ee37d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:21.777024 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0a56d667-91a1-457f-996c-c780e13c3fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:21.838532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1210/2420] 10.222.0.42 () {58 vars in 967 bytes} [Mon Sep 19 10:43:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:21.863825 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bb971-aa30-46fa-aa20-738c49cd888c service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:21.865445 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-907bb971-aa30-46fa-aa20-738c49cd888c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:21.938509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bb971-aa30-46fa-aa20-738c49cd888c service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:21.944319 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26857b-500e-4434-8378-865f94ce715f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17a8bc6c506c45be897bda23eaab485a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:21.944974 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26857b-500e-4434-8378-865f94ce715f service swift] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:21.945331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26857b-500e-4434-8378-865f94ce715f service swift] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:21.945645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26857b-500e-4434-8378-865f94ce715f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:21.955235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26857b-500e-4434-8378-865f94ce715f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:21.957051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26857b-500e-4434-8378-865f94ce715f service swift] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:21.965993 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1211/2421] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Sep 19 10:43:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 10:43:25.039918 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56d667-91a1-457f-996c-c780e13c3fdb None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:25.109306 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56d667-91a1-457f-996c-c780e13c3fdb None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:25.114185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e2fb5-bf3a-4524-9835-b4506190402f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:25.114816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e2fb5-bf3a-4524-9835-b4506190402f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:25.115099 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e2fb5-bf3a-4524-9835-b4506190402f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:25.115391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e2fb5-bf3a-4524-9835-b4506190402f admin admin] PATH_INFO: `/v3/users/29bcaf5bdc814d50ad48674876b9cf68` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:25.116520 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e2fb5-bf3a-4524-9835-b4506190402f admin admin] RBAC: Authorizing `identity:delete_user(user_id=29bcaf5bdc814d50ad48674876b9cf68)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:25.118458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e2fb5-bf3a-4524-9835-b4506190402f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:25.313315 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:25.322723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-842e2fb5-bf3a-4524-9835-b4506190402f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29bcaf5bdc814d50ad48674876b9cf68'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:25.323376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-842e2fb5-bf3a-4524-9835-b4506190402f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29bcaf5bdc814d50ad48674876b9cf68'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:25.410524 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-842e2fb5-bf3a-4524-9835-b4506190402f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29bcaf5bdc814d50ad48674876b9cf68'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:25.411204 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-842e2fb5-bf3a-4524-9835-b4506190402f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29bcaf5bdc814d50ad48674876b9cf68'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:25.424732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-842e2fb5-bf3a-4524-9835-b4506190402f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29bcaf5bdc814d50ad48674876b9cf68'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:25.437170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1211/2422] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:43:25 2022] DELETE /identity/v3/users/29bcaf5bdc814d50ad48674876b9cf68 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:25.441626 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf370c4a-1cc1-47e5-9a7d-dd52fc7e3587 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:25.442635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf370c4a-1cc1-47e5-9a7d-dd52fc7e3587 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:25.442723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf370c4a-1cc1-47e5-9a7d-dd52fc7e3587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:25.462617 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-cf370c4a-1cc1-47e5-9a7d-dd52fc7e3587 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:25.463810 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-cf370c4a-1cc1-47e5-9a7d-dd52fc7e3587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:25.526558 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1212/2423] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:25.536221 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e2fb5-bf3a-4524-9835-b4506190402f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:25.537766 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-842e2fb5-bf3a-4524-9835-b4506190402f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:25.597578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e2fb5-bf3a-4524-9835-b4506190402f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:25.601216 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f42974-5ecc-446a-b490-64f583bf2347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:25.601816 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f42974-5ecc-446a-b490-64f583bf2347 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:25.602094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f42974-5ecc-446a-b490-64f583bf2347 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:25.602364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f42974-5ecc-446a-b490-64f583bf2347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:25.611194 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f42974-5ecc-446a-b490-64f583bf2347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:25.612604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f42974-5ecc-446a-b490-64f583bf2347 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:25.622368 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1212/2424] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:43:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:43:26.023561 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf370c4a-1cc1-47e5-9a7d-dd52fc7e3587 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:26.025003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-cf370c4a-1cc1-47e5-9a7d-dd52fc7e3587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:26.037919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf370c4a-1cc1-47e5-9a7d-dd52fc7e3587 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:26.043391 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19073bd4-ca1b-4ea9-899c-129adbfb5f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:26.044079 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19073bd4-ca1b-4ea9-899c-129adbfb5f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:26.044404 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19073bd4-ca1b-4ea9-899c-129adbfb5f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:26.044723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19073bd4-ca1b-4ea9-899c-129adbfb5f88 admin admin] PATH_INFO: `/v3/projects/5f1a61ca4ecd4ea692b179f2de2d9694` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:26.045996 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19073bd4-ca1b-4ea9-899c-129adbfb5f88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f1a61ca4ecd4ea692b179f2de2d9694)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:26.048120 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19073bd4-ca1b-4ea9-899c-129adbfb5f88 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:26.155374 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:26.158514 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-19073bd4-ca1b-4ea9-899c-129adbfb5f88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f1a61ca4ecd4ea692b179f2de2d9694', 'request_id': 'req-19073bd4-ca1b-4ea9-899c-129adbfb5f88'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:26.165371 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-19073bd4-ca1b-4ea9-899c-129adbfb5f88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f1a61ca4ecd4ea692b179f2de2d9694', 'request_id': 'req-19073bd4-ca1b-4ea9-899c-129adbfb5f88'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:26.171770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-19073bd4-ca1b-4ea9-899c-129adbfb5f88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f1a61ca4ecd4ea692b179f2de2d9694', 'request_id': 'req-19073bd4-ca1b-4ea9-899c-129adbfb5f88'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:26.323664 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-19073bd4-ca1b-4ea9-899c-129adbfb5f88 admin admin] The token cache is being invalidate because project 5f1a61ca4ecd4ea692b179f2de2d9694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:43:26.323921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-19073bd4-ca1b-4ea9-899c-129adbfb5f88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:26.324590 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-19073bd4-ca1b-4ea9-899c-129adbfb5f88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:26.326132 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1213/2425] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:43:26 2022] DELETE /identity/v3/projects/5f1a61ca4ecd4ea692b179f2de2d9694 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:26.330152 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f42974-5ecc-446a-b490-64f583bf2347 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:26.341294 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f42974-5ecc-446a-b490-64f583bf2347 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:26.399029 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925ecff-35e0-4f01-98d1-be881ce7124b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:26.399656 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925ecff-35e0-4f01-98d1-be881ce7124b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:26.399936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925ecff-35e0-4f01-98d1-be881ce7124b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:26.400209 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925ecff-35e0-4f01-98d1-be881ce7124b admin admin] PATH_INFO: `/v3/users/8c6a23756d584438880a754507775f44` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:26.401355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925ecff-35e0-4f01-98d1-be881ce7124b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c6a23756d584438880a754507775f44)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:26.403284 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925ecff-35e0-4f01-98d1-be881ce7124b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:26.676759 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f925ecff-35e0-4f01-98d1-be881ce7124b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c6a23756d584438880a754507775f44'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:26.677569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f925ecff-35e0-4f01-98d1-be881ce7124b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c6a23756d584438880a754507775f44'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:26.792602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f925ecff-35e0-4f01-98d1-be881ce7124b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c6a23756d584438880a754507775f44'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:26.793288 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f925ecff-35e0-4f01-98d1-be881ce7124b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c6a23756d584438880a754507775f44'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:26.798872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:26.808153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f925ecff-35e0-4f01-98d1-be881ce7124b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c6a23756d584438880a754507775f44'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:26.820651 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1213/2426] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:43:26 2022] DELETE /identity/v3/users/8c6a23756d584438880a754507775f44 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:27.332375 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19073bd4-ca1b-4ea9-899c-129adbfb5f88 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:27.332375 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-19073bd4-ca1b-4ea9-899c-129adbfb5f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:27.349700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19073bd4-ca1b-4ea9-899c-129adbfb5f88 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:27.417673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbba6e-186d-4d2a-8520-9e1e20b83e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:27.418411 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbba6e-186d-4d2a-8520-9e1e20b83e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:27.418725 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbba6e-186d-4d2a-8520-9e1e20b83e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:27.419035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbba6e-186d-4d2a-8520-9e1e20b83e5c admin admin] PATH_INFO: `/v3/projects/78fe22a3084d4efbbd0e65f12ee7d61f` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:27.420365 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbba6e-186d-4d2a-8520-9e1e20b83e5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=78fe22a3084d4efbbd0e65f12ee7d61f)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:27.422720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbba6e-186d-4d2a-8520-9e1e20b83e5c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:27.562456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-24dbba6e-186d-4d2a-8520-9e1e20b83e5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78fe22a3084d4efbbd0e65f12ee7d61f', 'request_id': 'req-24dbba6e-186d-4d2a-8520-9e1e20b83e5c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:27.580325 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-24dbba6e-186d-4d2a-8520-9e1e20b83e5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78fe22a3084d4efbbd0e65f12ee7d61f', 'request_id': 'req-24dbba6e-186d-4d2a-8520-9e1e20b83e5c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:27.587733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-24dbba6e-186d-4d2a-8520-9e1e20b83e5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78fe22a3084d4efbbd0e65f12ee7d61f', 'request_id': 'req-24dbba6e-186d-4d2a-8520-9e1e20b83e5c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:27.662004 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-24dbba6e-186d-4d2a-8520-9e1e20b83e5c admin admin] The token cache is being invalidate because project 78fe22a3084d4efbbd0e65f12ee7d61f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:43:27.667714 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-24dbba6e-186d-4d2a-8520-9e1e20b83e5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:27.667714 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-24dbba6e-186d-4d2a-8520-9e1e20b83e5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:27.667714 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1214/2427] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:43:27 2022] DELETE /identity/v3/projects/78fe22a3084d4efbbd0e65f12ee7d61f => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:27.668315 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925ecff-35e0-4f01-98d1-be881ce7124b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:27.669737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f925ecff-35e0-4f01-98d1-be881ce7124b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:27.683517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925ecff-35e0-4f01-98d1-be881ce7124b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:27.748757 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cea47-3daa-4116-9cf4-5717dd72224b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:27.749432 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cea47-3daa-4116-9cf4-5717dd72224b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:27.749712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cea47-3daa-4116-9cf4-5717dd72224b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:27.749987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cea47-3daa-4116-9cf4-5717dd72224b admin admin] PATH_INFO: `/v3/users/d7d608cc8b32428aa13edd09caa3c82c` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:27.751171 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37cea47-3daa-4116-9cf4-5717dd72224b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7d608cc8b32428aa13edd09caa3c82c)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:27.753253 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37cea47-3daa-4116-9cf4-5717dd72224b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:27.995089 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c37cea47-3daa-4116-9cf4-5717dd72224b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7d608cc8b32428aa13edd09caa3c82c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:27.995853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c37cea47-3daa-4116-9cf4-5717dd72224b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7d608cc8b32428aa13edd09caa3c82c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:28.097403 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c37cea47-3daa-4116-9cf4-5717dd72224b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7d608cc8b32428aa13edd09caa3c82c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:28.097403 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c37cea47-3daa-4116-9cf4-5717dd72224b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7d608cc8b32428aa13edd09caa3c82c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:28.104973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c37cea47-3daa-4116-9cf4-5717dd72224b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7d608cc8b32428aa13edd09caa3c82c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:28.118659 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1214/2428] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:43:27 2022] DELETE /identity/v3/users/d7d608cc8b32428aa13edd09caa3c82c => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:28.623230 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbba6e-186d-4d2a-8520-9e1e20b83e5c admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:28.627672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-24dbba6e-186d-4d2a-8520-9e1e20b83e5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:28.638098 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbba6e-186d-4d2a-8520-9e1e20b83e5c admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:28.701731 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55d87a-f13d-498d-bc1b-b82c339c2f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:28.702361 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55d87a-f13d-498d-bc1b-b82c339c2f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:28.702635 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55d87a-f13d-498d-bc1b-b82c339c2f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:28.702906 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55d87a-f13d-498d-bc1b-b82c339c2f9b admin admin] PATH_INFO: `/v3/projects/98471785d59749aa9e676aa804572df4` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:28.704590 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55d87a-f13d-498d-bc1b-b82c339c2f9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=98471785d59749aa9e676aa804572df4)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:28.706700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55d87a-f13d-498d-bc1b-b82c339c2f9b admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:28.714735 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:28.845692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-db55d87a-f13d-498d-bc1b-b82c339c2f9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98471785d59749aa9e676aa804572df4', 'request_id': 'req-db55d87a-f13d-498d-bc1b-b82c339c2f9b'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:28.852220 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-db55d87a-f13d-498d-bc1b-b82c339c2f9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98471785d59749aa9e676aa804572df4', 'request_id': 'req-db55d87a-f13d-498d-bc1b-b82c339c2f9b'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:28.858885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-db55d87a-f13d-498d-bc1b-b82c339c2f9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98471785d59749aa9e676aa804572df4', 'request_id': 'req-db55d87a-f13d-498d-bc1b-b82c339c2f9b'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:28.935873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-db55d87a-f13d-498d-bc1b-b82c339c2f9b admin admin] The token cache is being invalidate because project 98471785d59749aa9e676aa804572df4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:43:28.936555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-db55d87a-f13d-498d-bc1b-b82c339c2f9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:28.937235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-db55d87a-f13d-498d-bc1b-b82c339c2f9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:28.938884 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1215/2429] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:43:28 2022] DELETE /identity/v3/projects/98471785d59749aa9e676aa804572df4 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:28.944957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d54a56-9e77-4b34-9b10-eac0ba908422 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:28.945541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d54a56-9e77-4b34-9b10-eac0ba908422 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:28.945541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d54a56-9e77-4b34-9b10-eac0ba908422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:28.965834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c4d54a56-9e77-4b34-9b10-eac0ba908422 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:28.967049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c4d54a56-9e77-4b34-9b10-eac0ba908422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:29.019063 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:29.033079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1215/2430] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:29.042034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2520361-3a9f-4254-9e46-253f72d684c2 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:29.042311 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2520361-3a9f-4254-9e46-253f72d684c2 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:29.042638 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2520361-3a9f-4254-9e46-253f72d684c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:29.063778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-a2520361-3a9f-4254-9e46-253f72d684c2 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:29.065150 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-a2520361-3a9f-4254-9e46-253f72d684c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:29.074170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1216/2431] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:29.081144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d54a56-9e77-4b34-9b10-eac0ba908422 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:29.089859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d54a56-9e77-4b34-9b10-eac0ba908422 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:29.094616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c850a6-24a8-4720-b277-683ecc81ca54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:29.095310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c850a6-24a8-4720-b277-683ecc81ca54 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:29.095659 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c850a6-24a8-4720-b277-683ecc81ca54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:29.096010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c850a6-24a8-4720-b277-683ecc81ca54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:29.096727 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c850a6-24a8-4720-b277-683ecc81ca54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:43:29.097133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c850a6-24a8-4720-b277-683ecc81ca54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:29.098845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c850a6-24a8-4720-b277-683ecc81ca54 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:29.111988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1216/2432] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:43:29.116374 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2520361-3a9f-4254-9e46-253f72d684c2 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:29.128664 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2520361-3a9f-4254-9e46-253f72d684c2 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:29.135197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24bb12-37df-4545-b5ce-bb2c64e6fb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:29.135836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24bb12-37df-4545-b5ce-bb2c64e6fb21 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:29.136115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24bb12-37df-4545-b5ce-bb2c64e6fb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:29.136438 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24bb12-37df-4545-b5ce-bb2c64e6fb21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:29.137093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24bb12-37df-4545-b5ce-bb2c64e6fb21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:29.139022 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24bb12-37df-4545-b5ce-bb2c64e6fb21 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:29.251804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1217/2433] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:43:29 2022] POST /identity/v3/projects => generated 363 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:43:29.256578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c850a6-24a8-4720-b277-683ecc81ca54 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:29.266594 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c850a6-24a8-4720-b277-683ecc81ca54 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:29.329220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d84d08-827e-4ce7-b6b4-b9e831683955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:29.329836 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d84d08-827e-4ce7-b6b4-b9e831683955 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:29.330111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d84d08-827e-4ce7-b6b4-b9e831683955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:29.330382 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d84d08-827e-4ce7-b6b4-b9e831683955 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:29.331096 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d84d08-827e-4ce7-b6b4-b9e831683955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:29.333325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d84d08-827e-4ce7-b6b4-b9e831683955 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:29.430027 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1217/2434] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:43:29 2022] POST /identity/v3/users => generated 282 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:43:29.431475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24bb12-37df-4545-b5ce-bb2c64e6fb21 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:29.447723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24bb12-37df-4545-b5ce-bb2c64e6fb21 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:29.461714 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147ee7b-f274-4082-9691-41d9c498c53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:29.462420 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147ee7b-f274-4082-9691-41d9c498c53f admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:29.462721 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147ee7b-f274-4082-9691-41d9c498c53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:29.463586 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147ee7b-f274-4082-9691-41d9c498c53f admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:29.464313 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147ee7b-f274-4082-9691-41d9c498c53f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:29.467350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147ee7b-f274-4082-9691-41d9c498c53f admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:29.486170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1218/2435] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:43:29.487444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d84d08-827e-4ce7-b6b4-b9e831683955 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:29.498936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d84d08-827e-4ce7-b6b4-b9e831683955 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:29.505133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d02b5-8b04-44db-a628-db3ac7767689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:29.505750 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d02b5-8b04-44db-a628-db3ac7767689 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:29.506022 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d02b5-8b04-44db-a628-db3ac7767689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:29.506374 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d02b5-8b04-44db-a628-db3ac7767689 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:29.507129 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d02b5-8b04-44db-a628-db3ac7767689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:29.508755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d02b5-8b04-44db-a628-db3ac7767689 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:29.519838 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1218/2436] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:29.524287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147ee7b-f274-4082-9691-41d9c498c53f admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:29.534444 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147ee7b-f274-4082-9691-41d9c498c53f admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:29.539198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa174530-ee0c-461e-aec7-db44ed6c80aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:29.539849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa174530-ee0c-461e-aec7-db44ed6c80aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:29.540126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa174530-ee0c-461e-aec7-db44ed6c80aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:29.540401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa174530-ee0c-461e-aec7-db44ed6c80aa admin admin] PATH_INFO: `/v3/projects/369e7b4c5eb9401dac0580e4bb4142d4/users/c9a74e34068a47a68cc4e1ed37835c2c/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:29.556788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa174530-ee0c-461e-aec7-db44ed6c80aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=369e7b4c5eb9401dac0580e4bb4142d4, user_id=c9a74e34068a47a68cc4e1ed37835c2c, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:29.559509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa174530-ee0c-461e-aec7-db44ed6c80aa admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:29.681043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1219/2437] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:43:29 2022] PUT /identity/v3/projects/369e7b4c5eb9401dac0580e4bb4142d4/users/c9a74e34068a47a68cc4e1ed37835c2c/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:29.747857 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295d97c-4a75-4809-9daf-813363dd3418 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:29.748170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295d97c-4a75-4809-9daf-813363dd3418 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:29.748527 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295d97c-4a75-4809-9daf-813363dd3418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:29.780982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b295d97c-4a75-4809-9daf-813363dd3418 None None] MFA Rules not processed for user `c9a74e34068a47a68cc4e1ed37835c2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:29.782528 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b295d97c-4a75-4809-9daf-813363dd3418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:29.849737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1219/2438] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:29.860770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa174530-ee0c-461e-aec7-db44ed6c80aa admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:29.862499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-aa174530-ee0c-461e-aec7-db44ed6c80aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:29.920819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa174530-ee0c-461e-aec7-db44ed6c80aa admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:29.924539 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65e1e0-e31e-4168-92ee-dcb62107f80d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:29.925184 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65e1e0-e31e-4168-92ee-dcb62107f80d service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:29.925494 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65e1e0-e31e-4168-92ee-dcb62107f80d service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:29.925771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65e1e0-e31e-4168-92ee-dcb62107f80d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:29.934586 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65e1e0-e31e-4168-92ee-dcb62107f80d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:29.936014 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65e1e0-e31e-4168-92ee-dcb62107f80d service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:29.945307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1220/2439] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:43:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:43:39.846586 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:39.940684 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:45.391211 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295d97c-4a75-4809-9daf-813363dd3418 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:45.392740 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b295d97c-4a75-4809-9daf-813363dd3418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:45.502602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295d97c-4a75-4809-9daf-813363dd3418 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:45.511231 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010527bd-00cb-478c-8beb-4ff949fe7170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:45.512355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010527bd-00cb-478c-8beb-4ff949fe7170 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:45.512904 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010527bd-00cb-478c-8beb-4ff949fe7170 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:45.513456 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010527bd-00cb-478c-8beb-4ff949fe7170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:45.526019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-010527bd-00cb-478c-8beb-4ff949fe7170 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:45.637126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010527bd-00cb-478c-8beb-4ff949fe7170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:45.638917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010527bd-00cb-478c-8beb-4ff949fe7170 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:45.653283 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1220/2440] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:43:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:43:49.857981 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65e1e0-e31e-4168-92ee-dcb62107f80d service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:49.857981 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5e65e1e0-e31e-4168-92ee-dcb62107f80d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:49.876168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65e1e0-e31e-4168-92ee-dcb62107f80d service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:49.882342 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a6ec0-f9fc-46b6-b819-12dab03e8461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:49.883126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a6ec0-f9fc-46b6-b819-12dab03e8461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:49.883426 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a6ec0-f9fc-46b6-b819-12dab03e8461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:49.883726 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a6ec0-f9fc-46b6-b819-12dab03e8461 admin admin] PATH_INFO: `/v3/users/6712334ebb77462daa89be58d4480846` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:49.884971 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a6ec0-f9fc-46b6-b819-12dab03e8461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6712334ebb77462daa89be58d4480846)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:49.887112 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a6ec0-f9fc-46b6-b819-12dab03e8461 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:50.118209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-609a6ec0-f9fc-46b6-b819-12dab03e8461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6712334ebb77462daa89be58d4480846'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:50.221847 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-609a6ec0-f9fc-46b6-b819-12dab03e8461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6712334ebb77462daa89be58d4480846'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:50.222608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-609a6ec0-f9fc-46b6-b819-12dab03e8461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6712334ebb77462daa89be58d4480846'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:50.223868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-609a6ec0-f9fc-46b6-b819-12dab03e8461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6712334ebb77462daa89be58d4480846'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:50.232025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-609a6ec0-f9fc-46b6-b819-12dab03e8461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6712334ebb77462daa89be58d4480846'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:50.251794 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1221/2441] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:43:49 2022] DELETE /identity/v3/users/6712334ebb77462daa89be58d4480846 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:50.531922 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010527bd-00cb-478c-8beb-4ff949fe7170 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:50.544803 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010527bd-00cb-478c-8beb-4ff949fe7170 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:50.611293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1017b85-cdda-4256-a694-03dc8af6bfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:50.611977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1017b85-cdda-4256-a694-03dc8af6bfa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:50.612205 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1017b85-cdda-4256-a694-03dc8af6bfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:50.612561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1017b85-cdda-4256-a694-03dc8af6bfa0 admin admin] PATH_INFO: `/v3/projects/67ec80c3417443d7b2368506bbd4a814` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:50.613730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1017b85-cdda-4256-a694-03dc8af6bfa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67ec80c3417443d7b2368506bbd4a814)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:50.615730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1017b85-cdda-4256-a694-03dc8af6bfa0 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:50.719573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a1017b85-cdda-4256-a694-03dc8af6bfa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67ec80c3417443d7b2368506bbd4a814', 'request_id': 'req-a1017b85-cdda-4256-a694-03dc8af6bfa0'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:50.721808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:50.726426 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a1017b85-cdda-4256-a694-03dc8af6bfa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67ec80c3417443d7b2368506bbd4a814', 'request_id': 'req-a1017b85-cdda-4256-a694-03dc8af6bfa0'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:50.732797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a1017b85-cdda-4256-a694-03dc8af6bfa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67ec80c3417443d7b2368506bbd4a814', 'request_id': 'req-a1017b85-cdda-4256-a694-03dc8af6bfa0'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:50.866530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a1017b85-cdda-4256-a694-03dc8af6bfa0 admin admin] The token cache is being invalidate because project 67ec80c3417443d7b2368506bbd4a814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:43:50.866723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a1017b85-cdda-4256-a694-03dc8af6bfa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:50.867345 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a1017b85-cdda-4256-a694-03dc8af6bfa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:50.868802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1221/2442] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:43:50 2022] DELETE /identity/v3/projects/67ec80c3417443d7b2368506bbd4a814 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:43:50.873795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04794017-d9df-4afd-b1a0-c3f95b4dbbf4 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:50.874273 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04794017-d9df-4afd-b1a0-c3f95b4dbbf4 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:50.874368 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04794017-d9df-4afd-b1a0-c3f95b4dbbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:50.903085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-04794017-d9df-4afd-b1a0-c3f95b4dbbf4 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:50.904984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-04794017-d9df-4afd-b1a0-c3f95b4dbbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:50.993155 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1222/2443] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:51.002460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d41b74-bcf9-4153-aaf8-977e8b02e5b0 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:51.002736 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d41b74-bcf9-4153-aaf8-977e8b02e5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:51.003004 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d41b74-bcf9-4153-aaf8-977e8b02e5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:51.024044 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-55d41b74-bcf9-4153-aaf8-977e8b02e5b0 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:51.025407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-55d41b74-bcf9-4153-aaf8-977e8b02e5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:51.033354 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1222/2444] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:51.037742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04794017-d9df-4afd-b1a0-c3f95b4dbbf4 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:51.048575 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04794017-d9df-4afd-b1a0-c3f95b4dbbf4 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:51.053671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1510a-264d-4343-afbe-1ef6d6aac18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:51.054220 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1510a-264d-4343-afbe-1ef6d6aac18b admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:51.054498 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1510a-264d-4343-afbe-1ef6d6aac18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:51.054769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1510a-264d-4343-afbe-1ef6d6aac18b admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:51.055471 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1510a-264d-4343-afbe-1ef6d6aac18b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:43:51.055820 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1510a-264d-4343-afbe-1ef6d6aac18b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:51.057425 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1510a-264d-4343-afbe-1ef6d6aac18b admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:51.070420 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1223/2445] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:43:51.074950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d41b74-bcf9-4153-aaf8-977e8b02e5b0 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:51.084811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d41b74-bcf9-4153-aaf8-977e8b02e5b0 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:51.089575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76583ad-6af2-45cf-b713-2180acd62277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:51.090106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76583ad-6af2-45cf-b713-2180acd62277 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:51.090380 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76583ad-6af2-45cf-b713-2180acd62277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:51.090671 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76583ad-6af2-45cf-b713-2180acd62277 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:51.091308 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76583ad-6af2-45cf-b713-2180acd62277 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:51.093213 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76583ad-6af2-45cf-b713-2180acd62277 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:51.178987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1223/2446] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:43:51 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:43:51.185305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1510a-264d-4343-afbe-1ef6d6aac18b admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:51.198700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1510a-264d-4343-afbe-1ef6d6aac18b admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:51.305828 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f93cb-d6d4-4cb1-9936-2b714ee43e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:51.306547 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f93cb-d6d4-4cb1-9936-2b714ee43e50 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:51.306852 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f93cb-d6d4-4cb1-9936-2b714ee43e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:51.307133 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f93cb-d6d4-4cb1-9936-2b714ee43e50 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:51.307757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f93cb-d6d4-4cb1-9936-2b714ee43e50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:51.309739 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f93cb-d6d4-4cb1-9936-2b714ee43e50 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:51.318059 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:51.416529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1224/2447] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:43:51 2022] POST /identity/v3/users => generated 292 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:43:51.421456 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76583ad-6af2-45cf-b713-2180acd62277 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:51.433677 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76583ad-6af2-45cf-b713-2180acd62277 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:51.439364 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793da33-bd8c-49bf-9d9f-e006db1a3df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:51.440040 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793da33-bd8c-49bf-9d9f-e006db1a3df2 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:51.440363 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793da33-bd8c-49bf-9d9f-e006db1a3df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:51.440704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793da33-bd8c-49bf-9d9f-e006db1a3df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:51.441501 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793da33-bd8c-49bf-9d9f-e006db1a3df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:51.443386 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793da33-bd8c-49bf-9d9f-e006db1a3df2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:51.455171 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1224/2448] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:51.460304 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f93cb-d6d4-4cb1-9936-2b714ee43e50 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:51.471413 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f93cb-d6d4-4cb1-9936-2b714ee43e50 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:51.478158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b9fc6-2a18-49f0-9bfb-67fb04ae0a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:51.478950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b9fc6-2a18-49f0-9bfb-67fb04ae0a32 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:51.479351 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b9fc6-2a18-49f0-9bfb-67fb04ae0a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:51.479756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b9fc6-2a18-49f0-9bfb-67fb04ae0a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:51.480698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b9fc6-2a18-49f0-9bfb-67fb04ae0a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:51.482664 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b9fc6-2a18-49f0-9bfb-67fb04ae0a32 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:51.496966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1225/2449] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:51.510634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793da33-bd8c-49bf-9d9f-e006db1a3df2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:51.521750 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793da33-bd8c-49bf-9d9f-e006db1a3df2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:51.526751 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0237de4-ecdd-4193-b102-cd07a8919674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:51.527432 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0237de4-ecdd-4193-b102-cd07a8919674 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:51.527708 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0237de4-ecdd-4193-b102-cd07a8919674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:51.527978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0237de4-ecdd-4193-b102-cd07a8919674 admin admin] PATH_INFO: `/v3/projects/e4413bdac283489b86d9059967e509a6/users/8ac49bfdb25344daaecd623a1b8ef9ad/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:51.545695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0237de4-ecdd-4193-b102-cd07a8919674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4413bdac283489b86d9059967e509a6, user_id=8ac49bfdb25344daaecd623a1b8ef9ad, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:51.549499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0237de4-ecdd-4193-b102-cd07a8919674 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:51.646221 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1225/2450] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:43:51 2022] PUT /identity/v3/projects/e4413bdac283489b86d9059967e509a6/users/8ac49bfdb25344daaecd623a1b8ef9ad/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:51.656672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed2016-d33f-42b1-8c68-552833b86a20 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:51.656951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed2016-d33f-42b1-8c68-552833b86a20 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:51.657254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed2016-d33f-42b1-8c68-552833b86a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:51.685024 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-80ed2016-d33f-42b1-8c68-552833b86a20 None None] MFA Rules not processed for user `8ac49bfdb25344daaecd623a1b8ef9ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:51.686232 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-80ed2016-d33f-42b1-8c68-552833b86a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:51.751230 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1226/2451] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:43:51.762467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0237de4-ecdd-4193-b102-cd07a8919674 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:51.764489 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a0237de4-ecdd-4193-b102-cd07a8919674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:51.845201 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0237de4-ecdd-4193-b102-cd07a8919674 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:51.851336 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef95669-cd50-4781-aec3-e8cbb09a262d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:51.852061 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef95669-cd50-4781-aec3-e8cbb09a262d service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:51.852286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef95669-cd50-4781-aec3-e8cbb09a262d service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:51.852575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef95669-cd50-4781-aec3-e8cbb09a262d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:51.860885 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef95669-cd50-4781-aec3-e8cbb09a262d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:51.862713 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef95669-cd50-4781-aec3-e8cbb09a262d service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:51.872207 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1226/2452] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:43:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:43:52.721789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed2016-d33f-42b1-8c68-552833b86a20 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:52.723784 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-80ed2016-d33f-42b1-8c68-552833b86a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:52.784187 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed2016-d33f-42b1-8c68-552833b86a20 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:52.788591 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c998b36-c19c-4a4c-9e89-4075625bb8b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:52.789225 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c998b36-c19c-4a4c-9e89-4075625bb8b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:52.789503 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c998b36-c19c-4a4c-9e89-4075625bb8b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:52.789771 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c998b36-c19c-4a4c-9e89-4075625bb8b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:52.791733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-7c998b36-c19c-4a4c-9e89-4075625bb8b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:52.863815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c998b36-c19c-4a4c-9e89-4075625bb8b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:52.865401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c998b36-c19c-4a4c-9e89-4075625bb8b7 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:52.875327 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1227/2453] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:43:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:43:56.879037 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef95669-cd50-4781-aec3-e8cbb09a262d service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:56.880636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9ef95669-cd50-4781-aec3-e8cbb09a262d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:56.895310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef95669-cd50-4781-aec3-e8cbb09a262d service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:56.900206 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98b97c-43c4-4cd9-b7ce-abd7b506bb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:56.900831 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98b97c-43c4-4cd9-b7ce-abd7b506bb42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:56.901142 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98b97c-43c4-4cd9-b7ce-abd7b506bb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:56.901416 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98b97c-43c4-4cd9-b7ce-abd7b506bb42 admin admin] PATH_INFO: `/v3/users/8a059ba74d744a12a4694513f8ca5789` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:56.902556 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98b97c-43c4-4cd9-b7ce-abd7b506bb42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a059ba74d744a12a4694513f8ca5789)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:56.904478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98b97c-43c4-4cd9-b7ce-abd7b506bb42 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:57.043236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9e98b97c-43c4-4cd9-b7ce-abd7b506bb42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a059ba74d744a12a4694513f8ca5789'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:57.043915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9e98b97c-43c4-4cd9-b7ce-abd7b506bb42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a059ba74d744a12a4694513f8ca5789'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:57.112618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9e98b97c-43c4-4cd9-b7ce-abd7b506bb42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a059ba74d744a12a4694513f8ca5789'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:57.113838 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9e98b97c-43c4-4cd9-b7ce-abd7b506bb42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a059ba74d744a12a4694513f8ca5789'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:57.116380 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:57.127616 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-9e98b97c-43c4-4cd9-b7ce-abd7b506bb42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a059ba74d744a12a4694513f8ca5789'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:57.141410 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1227/2454] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:43:56 2022] DELETE /identity/v3/users/8a059ba74d744a12a4694513f8ca5789 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:57.378167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c998b36-c19c-4a4c-9e89-4075625bb8b7 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:57.388311 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c998b36-c19c-4a4c-9e89-4075625bb8b7 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:57.401651 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:57.445030 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bf9fc-244a-4554-9044-50e2938351fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:57.445683 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bf9fc-244a-4554-9044-50e2938351fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:57.445960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bf9fc-244a-4554-9044-50e2938351fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:57.446230 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bf9fc-244a-4554-9044-50e2938351fc admin admin] PATH_INFO: `/v3/projects/e86736f46b254bc399298305a8888467` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:57.447358 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7bf9fc-244a-4554-9044-50e2938351fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e86736f46b254bc399298305a8888467)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:57.449299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7bf9fc-244a-4554-9044-50e2938351fc admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:57.587891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7c7bf9fc-244a-4554-9044-50e2938351fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e86736f46b254bc399298305a8888467', 'request_id': 'req-7c7bf9fc-244a-4554-9044-50e2938351fc'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:57.593633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7c7bf9fc-244a-4554-9044-50e2938351fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e86736f46b254bc399298305a8888467', 'request_id': 'req-7c7bf9fc-244a-4554-9044-50e2938351fc'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:57.600111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7c7bf9fc-244a-4554-9044-50e2938351fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e86736f46b254bc399298305a8888467', 'request_id': 'req-7c7bf9fc-244a-4554-9044-50e2938351fc'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:57.696350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7c7bf9fc-244a-4554-9044-50e2938351fc admin admin] The token cache is being invalidate because project e86736f46b254bc399298305a8888467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:43:57.696750 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7c7bf9fc-244a-4554-9044-50e2938351fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:57.697499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7c7bf9fc-244a-4554-9044-50e2938351fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:43:57.699579 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1228/2455] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:43:57 2022] DELETE /identity/v3/projects/e86736f46b254bc399298305a8888467 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:57.711721 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc555ac9-bc47-444e-abff-c12f9123c6b2 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:57.712026 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc555ac9-bc47-444e-abff-c12f9123c6b2 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:57.712305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc555ac9-bc47-444e-abff-c12f9123c6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:57.733723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-bc555ac9-bc47-444e-abff-c12f9123c6b2 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:57.734981 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bc555ac9-bc47-444e-abff-c12f9123c6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:57.798219 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1228/2456] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:57.807130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78dd2a-d9b5-4c5b-9d64-43e44a78dc89 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:57.807412 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78dd2a-d9b5-4c5b-9d64-43e44a78dc89 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:57.807690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78dd2a-d9b5-4c5b-9d64-43e44a78dc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:57.828187 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-fd78dd2a-d9b5-4c5b-9d64-43e44a78dc89 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:57.829486 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-fd78dd2a-d9b5-4c5b-9d64-43e44a78dc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:57.838373 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1229/2457] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:57.843038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc555ac9-bc47-444e-abff-c12f9123c6b2 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:57.854723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc555ac9-bc47-444e-abff-c12f9123c6b2 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:57.860342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015e9dc-ea61-4a4b-8a69-e4b1a2c0eeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:57.861042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015e9dc-ea61-4a4b-8a69-e4b1a2c0eeb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:57.861397 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015e9dc-ea61-4a4b-8a69-e4b1a2c0eeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:57.861680 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015e9dc-ea61-4a4b-8a69-e4b1a2c0eeb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:57.869662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015e9dc-ea61-4a4b-8a69-e4b1a2c0eeb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:43:57.869662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015e9dc-ea61-4a4b-8a69-e4b1a2c0eeb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:57.869662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015e9dc-ea61-4a4b-8a69-e4b1a2c0eeb7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:57.877865 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1229/2458] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:43:57.883129 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78dd2a-d9b5-4c5b-9d64-43e44a78dc89 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:57.896749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78dd2a-d9b5-4c5b-9d64-43e44a78dc89 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:57.907461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3050ef-d048-40f2-8914-913ad32d6bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:57.908132 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3050ef-d048-40f2-8914-913ad32d6bac admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:57.908449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3050ef-d048-40f2-8914-913ad32d6bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:57.908770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3050ef-d048-40f2-8914-913ad32d6bac admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:57.909969 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3050ef-d048-40f2-8914-913ad32d6bac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:57.912219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3050ef-d048-40f2-8914-913ad32d6bac admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:58.012836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1230/2459] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:43:57 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:43:58.017350 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015e9dc-ea61-4a4b-8a69-e4b1a2c0eeb7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:58.029379 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015e9dc-ea61-4a4b-8a69-e4b1a2c0eeb7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:58.091185 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f555690-0a7e-4135-be2a-71bb80dd9eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:58.091814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f555690-0a7e-4135-be2a-71bb80dd9eab admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:58.092098 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f555690-0a7e-4135-be2a-71bb80dd9eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:58.092380 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f555690-0a7e-4135-be2a-71bb80dd9eab admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:58.093102 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f555690-0a7e-4135-be2a-71bb80dd9eab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:58.095652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f555690-0a7e-4135-be2a-71bb80dd9eab admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:58.274856 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1230/2460] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:43:58 2022] POST /identity/v3/users => generated 285 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:43:58.279069 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3050ef-d048-40f2-8914-913ad32d6bac admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:58.289613 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3050ef-d048-40f2-8914-913ad32d6bac admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:58.294237 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44019f84-d415-4ccc-8fe7-94cddac22ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:58.294855 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44019f84-d415-4ccc-8fe7-94cddac22ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:58.295153 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44019f84-d415-4ccc-8fe7-94cddac22ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:58.295445 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44019f84-d415-4ccc-8fe7-94cddac22ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:58.296131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44019f84-d415-4ccc-8fe7-94cddac22ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:58.297799 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44019f84-d415-4ccc-8fe7-94cddac22ed6 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:58.308331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1231/2461] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:58.312472 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f555690-0a7e-4135-be2a-71bb80dd9eab admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:58.323226 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f555690-0a7e-4135-be2a-71bb80dd9eab admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:58.327806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415711d-4d1f-443b-a46a-ab4ec1f9297b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:58.328401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415711d-4d1f-443b-a46a-ab4ec1f9297b admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:58.328680 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415711d-4d1f-443b-a46a-ab4ec1f9297b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:58.328957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415711d-4d1f-443b-a46a-ab4ec1f9297b admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:58.329642 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415711d-4d1f-443b-a46a-ab4ec1f9297b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:58.331223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415711d-4d1f-443b-a46a-ab4ec1f9297b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:58.343013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1231/2462] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:58.347182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44019f84-d415-4ccc-8fe7-94cddac22ed6 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:58.357257 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44019f84-d415-4ccc-8fe7-94cddac22ed6 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:58.361777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ed053-00f7-4478-8a6e-c2d93d2605ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:58.362416 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ed053-00f7-4478-8a6e-c2d93d2605ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:58.362692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ed053-00f7-4478-8a6e-c2d93d2605ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:58.362963 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ed053-00f7-4478-8a6e-c2d93d2605ff admin admin] PATH_INFO: `/v3/projects/f96ae4bdab0349ed859c7ae8aa59ac6b/users/498f068f5ed245e3aed418d31c9dd08f/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:58.377356 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ed053-00f7-4478-8a6e-c2d93d2605ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96ae4bdab0349ed859c7ae8aa59ac6b, user_id=498f068f5ed245e3aed418d31c9dd08f, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:58.380509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ed053-00f7-4478-8a6e-c2d93d2605ff admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:58.474239 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1232/2463] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:43:58 2022] PUT /identity/v3/projects/f96ae4bdab0349ed859c7ae8aa59ac6b/users/498f068f5ed245e3aed418d31c9dd08f/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:58.485829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737e29d-bf45-4400-8888-1a355ae55629 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:58.487842 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737e29d-bf45-4400-8888-1a355ae55629 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:58.487842 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737e29d-bf45-4400-8888-1a355ae55629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:58.516547 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a737e29d-bf45-4400-8888-1a355ae55629 None None] MFA Rules not processed for user `498f068f5ed245e3aed418d31c9dd08f`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:58.517784 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a737e29d-bf45-4400-8888-1a355ae55629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:58.546357 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:58.585892 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1232/2464] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:58.590749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ed053-00f7-4478-8a6e-c2d93d2605ff admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:58.637913 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:58.655495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ed053-00f7-4478-8a6e-c2d93d2605ff admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:58.660745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1152747-8042-4a6d-b22d-f4f0923ae10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:58.661377 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1152747-8042-4a6d-b22d-f4f0923ae10a admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:58.661658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1152747-8042-4a6d-b22d-f4f0923ae10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:58.661931 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1152747-8042-4a6d-b22d-f4f0923ae10a admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:58.662564 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1152747-8042-4a6d-b22d-f4f0923ae10a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:58.664526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1152747-8042-4a6d-b22d-f4f0923ae10a admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:58.808132 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1233/2465] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:43:58 2022] POST /identity/v3/projects => generated 367 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:43:58.812323 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737e29d-bf45-4400-8888-1a355ae55629 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:58.824527 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737e29d-bf45-4400-8888-1a355ae55629 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:58.902376 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc97d4-076e-43d4-9393-bcc92f82ecb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:58.902990 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc97d4-076e-43d4-9393-bcc92f82ecb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:58.903266 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc97d4-076e-43d4-9393-bcc92f82ecb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:58.903549 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc97d4-076e-43d4-9393-bcc92f82ecb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:58.904183 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc97d4-076e-43d4-9393-bcc92f82ecb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:58.906292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc97d4-076e-43d4-9393-bcc92f82ecb6 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:59.073775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1233/2466] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:43:58 2022] POST /identity/v3/users => generated 290 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:43:59.077924 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1152747-8042-4a6d-b22d-f4f0923ae10a admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:59.089090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1152747-8042-4a6d-b22d-f4f0923ae10a admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:59.093673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc55370-4f0b-4538-b1d8-dc46366c1483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:59.094284 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc55370-4f0b-4538-b1d8-dc46366c1483 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:59.094560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc55370-4f0b-4538-b1d8-dc46366c1483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:59.094832 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc55370-4f0b-4538-b1d8-dc46366c1483 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:59.095471 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc55370-4f0b-4538-b1d8-dc46366c1483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:59.097080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc55370-4f0b-4538-b1d8-dc46366c1483 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:59.107279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1234/2467] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:43:59.112182 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc97d4-076e-43d4-9393-bcc92f82ecb6 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:59.123956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc97d4-076e-43d4-9393-bcc92f82ecb6 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:59.129644 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654324e-815f-4d11-84c2-819973fb51a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:59.130363 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654324e-815f-4d11-84c2-819973fb51a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:59.130668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654324e-815f-4d11-84c2-819973fb51a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:59.130967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654324e-815f-4d11-84c2-819973fb51a1 admin admin] PATH_INFO: `/v3/projects/bd61355891c74819b1194dd07026a9b2/users/e0255aa04e7e42539a357c57187b1eca/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:59.147696 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654324e-815f-4d11-84c2-819973fb51a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd61355891c74819b1194dd07026a9b2, user_id=e0255aa04e7e42539a357c57187b1eca, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:59.150725 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654324e-815f-4d11-84c2-819973fb51a1 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:59.239758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1234/2468] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:43:59 2022] PUT /identity/v3/projects/bd61355891c74819b1194dd07026a9b2/users/e0255aa04e7e42539a357c57187b1eca/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:59.244423 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc55370-4f0b-4538-b1d8-dc46366c1483 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:59.255137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc55370-4f0b-4538-b1d8-dc46366c1483 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:59.317087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafd48c-fb2f-4f2b-9469-a475fb6ab26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:59.317891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafd48c-fb2f-4f2b-9469-a475fb6ab26d admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:59.318197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafd48c-fb2f-4f2b-9469-a475fb6ab26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:59.318494 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafd48c-fb2f-4f2b-9469-a475fb6ab26d admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:59.319214 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafd48c-fb2f-4f2b-9469-a475fb6ab26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:59.321209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafd48c-fb2f-4f2b-9469-a475fb6ab26d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:59.333901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1235/2469] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:43:59.338320 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654324e-815f-4d11-84c2-819973fb51a1 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:59.349505 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654324e-815f-4d11-84c2-819973fb51a1 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:59.354973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca645da-a3cb-4ac3-8b0a-eae5840f1a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:59.356078 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca645da-a3cb-4ac3-8b0a-eae5840f1a74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:59.356444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca645da-a3cb-4ac3-8b0a-eae5840f1a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:59.356784 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca645da-a3cb-4ac3-8b0a-eae5840f1a74 admin admin] PATH_INFO: `/v3/projects/bd61355891c74819b1194dd07026a9b2/users/e0255aa04e7e42539a357c57187b1eca/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:59.359775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca645da-a3cb-4ac3-8b0a-eae5840f1a74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd61355891c74819b1194dd07026a9b2, user_id=e0255aa04e7e42539a357c57187b1eca, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:59.364209 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca645da-a3cb-4ac3-8b0a-eae5840f1a74 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:59.376092 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1235/2470] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:43:59 2022] PUT /identity/v3/projects/bd61355891c74819b1194dd07026a9b2/users/e0255aa04e7e42539a357c57187b1eca/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:43:59.387273 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdb94d-3945-4a63-9644-c28c956612ee None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:59.387827 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdb94d-3945-4a63-9644-c28c956612ee None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:59.387827 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdb94d-3945-4a63-9644-c28c956612ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:59.417670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-5efdb94d-3945-4a63-9644-c28c956612ee None None] MFA Rules not processed for user `e0255aa04e7e42539a357c57187b1eca`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:43:59.418876 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5efdb94d-3945-4a63-9644-c28c956612ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:59.490320 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1236/2471] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:43:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:43:59.502764 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca645da-a3cb-4ac3-8b0a-eae5840f1a74 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:59.505242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-fca645da-a3cb-4ac3-8b0a-eae5840f1a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:59.566692 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca645da-a3cb-4ac3-8b0a-eae5840f1a74 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:59.570347 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991268a-b3be-4d7f-be49-86d4ea75afde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:59.570953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991268a-b3be-4d7f-be49-86d4ea75afde service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:59.571231 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991268a-b3be-4d7f-be49-86d4ea75afde service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:59.571505 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991268a-b3be-4d7f-be49-86d4ea75afde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:59.582429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991268a-b3be-4d7f-be49-86d4ea75afde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:59.583965 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991268a-b3be-4d7f-be49-86d4ea75afde service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:59.594026 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1236/2472] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:43:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:43:59.751976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdb94d-3945-4a63-9644-c28c956612ee None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:43:59.757372 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5efdb94d-3945-4a63-9644-c28c956612ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:43:59.766519 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdb94d-3945-4a63-9644-c28c956612ee None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:43:59.771091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4285d-68ae-4a0e-867c-213dc538b23d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:43:59.771773 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4285d-68ae-4a0e-867c-213dc538b23d service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:43:59.772052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4285d-68ae-4a0e-867c-213dc538b23d service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:43:59.772322 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4285d-68ae-4a0e-867c-213dc538b23d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:43:59.815671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:43:59.846194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4285d-68ae-4a0e-867c-213dc538b23d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:43:59.847591 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4285d-68ae-4a0e-867c-213dc538b23d service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:43:59.856790 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1237/2473] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:43:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:44:01.452385 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991268a-b3be-4d7f-be49-86d4ea75afde service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:01.452385 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1991268a-b3be-4d7f-be49-86d4ea75afde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:01.482707 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:01.524413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991268a-b3be-4d7f-be49-86d4ea75afde service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:01.529615 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82ec2c-2ddd-4b1b-9ecd-fb578a31f684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:01.530329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82ec2c-2ddd-4b1b-9ecd-fb578a31f684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:01.530670 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82ec2c-2ddd-4b1b-9ecd-fb578a31f684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:01.530972 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82ec2c-2ddd-4b1b-9ecd-fb578a31f684 admin admin] PATH_INFO: `/v3/users/8ac49bfdb25344daaecd623a1b8ef9ad` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:01.532265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82ec2c-2ddd-4b1b-9ecd-fb578a31f684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ac49bfdb25344daaecd623a1b8ef9ad)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:01.534539 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82ec2c-2ddd-4b1b-9ecd-fb578a31f684 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:01.740996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3a82ec2c-2ddd-4b1b-9ecd-fb578a31f684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ac49bfdb25344daaecd623a1b8ef9ad'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:01.741742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3a82ec2c-2ddd-4b1b-9ecd-fb578a31f684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ac49bfdb25344daaecd623a1b8ef9ad'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:01.825760 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3a82ec2c-2ddd-4b1b-9ecd-fb578a31f684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ac49bfdb25344daaecd623a1b8ef9ad'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:01.826607 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3a82ec2c-2ddd-4b1b-9ecd-fb578a31f684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ac49bfdb25344daaecd623a1b8ef9ad'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:01.843075 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3a82ec2c-2ddd-4b1b-9ecd-fb578a31f684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ac49bfdb25344daaecd623a1b8ef9ad'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:01.857801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1237/2474] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:44:01 2022] DELETE /identity/v3/users/8ac49bfdb25344daaecd623a1b8ef9ad => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:01.862822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6739b-1cb2-4c3f-9011-44ee22ce8a3b None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:01.863328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6739b-1cb2-4c3f-9011-44ee22ce8a3b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:01.863426 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6739b-1cb2-4c3f-9011-44ee22ce8a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:01.885347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-e7c6739b-1cb2-4c3f-9011-44ee22ce8a3b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:01.886706 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e7c6739b-1cb2-4c3f-9011-44ee22ce8a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:01.964680 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1238/2475] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:01.975736 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82ec2c-2ddd-4b1b-9ecd-fb578a31f684 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:01.977170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3a82ec2c-2ddd-4b1b-9ecd-fb578a31f684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:02.038213 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82ec2c-2ddd-4b1b-9ecd-fb578a31f684 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:02.042144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce940999-1deb-4d8e-8280-9380909b7194 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:02.042763 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce940999-1deb-4d8e-8280-9380909b7194 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:02.043066 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce940999-1deb-4d8e-8280-9380909b7194 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:02.043338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce940999-1deb-4d8e-8280-9380909b7194 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:02.054210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce940999-1deb-4d8e-8280-9380909b7194 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:02.055764 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce940999-1deb-4d8e-8280-9380909b7194 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:02.067421 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1238/2476] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:44:02.546391 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6739b-1cb2-4c3f-9011-44ee22ce8a3b None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:02.548574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e7c6739b-1cb2-4c3f-9011-44ee22ce8a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:02.564228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6739b-1cb2-4c3f-9011-44ee22ce8a3b None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:02.570079 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d0ce8-7c4b-42d2-b26e-21ff48c2ac6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:02.570732 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d0ce8-7c4b-42d2-b26e-21ff48c2ac6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:02.572375 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d0ce8-7c4b-42d2-b26e-21ff48c2ac6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:02.572375 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d0ce8-7c4b-42d2-b26e-21ff48c2ac6a admin admin] PATH_INFO: `/v3/projects/e4413bdac283489b86d9059967e509a6` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:02.572600 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d0ce8-7c4b-42d2-b26e-21ff48c2ac6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4413bdac283489b86d9059967e509a6)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:02.575517 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d0ce8-7c4b-42d2-b26e-21ff48c2ac6a admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:02.708528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-321d0ce8-7c4b-42d2-b26e-21ff48c2ac6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4413bdac283489b86d9059967e509a6', 'request_id': 'req-321d0ce8-7c4b-42d2-b26e-21ff48c2ac6a'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:02.714899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-321d0ce8-7c4b-42d2-b26e-21ff48c2ac6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4413bdac283489b86d9059967e509a6', 'request_id': 'req-321d0ce8-7c4b-42d2-b26e-21ff48c2ac6a'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:02.721937 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-321d0ce8-7c4b-42d2-b26e-21ff48c2ac6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4413bdac283489b86d9059967e509a6', 'request_id': 'req-321d0ce8-7c4b-42d2-b26e-21ff48c2ac6a'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:02.861548 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-321d0ce8-7c4b-42d2-b26e-21ff48c2ac6a admin admin] The token cache is being invalidate because project e4413bdac283489b86d9059967e509a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:44:02.861831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-321d0ce8-7c4b-42d2-b26e-21ff48c2ac6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:02.862465 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-321d0ce8-7c4b-42d2-b26e-21ff48c2ac6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:02.864001 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1239/2477] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:44:02 2022] DELETE /identity/v3/projects/e4413bdac283489b86d9059967e509a6 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:02.870645 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de65064-af04-40cb-a97e-ab202f1f978e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:02.871062 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de65064-af04-40cb-a97e-ab202f1f978e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:02.871350 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de65064-af04-40cb-a97e-ab202f1f978e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:02.893043 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1de65064-af04-40cb-a97e-ab202f1f978e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:02.894356 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1de65064-af04-40cb-a97e-ab202f1f978e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:02.939896 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:02.959323 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1239/2478] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:02.967815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252a379-62b5-40f6-88d7-50af41747b0a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:02.968148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252a379-62b5-40f6-88d7-50af41747b0a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:02.968859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252a379-62b5-40f6-88d7-50af41747b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:02.990038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-d252a379-62b5-40f6-88d7-50af41747b0a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:03.006332 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d252a379-62b5-40f6-88d7-50af41747b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:03.006332 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1240/2479] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:03.007409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de65064-af04-40cb-a97e-ab202f1f978e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:03.017139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de65064-af04-40cb-a97e-ab202f1f978e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:03.023487 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211382f-39ea-4ee5-bbd0-60d008969f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:03.024173 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211382f-39ea-4ee5-bbd0-60d008969f6c admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:03.024534 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211382f-39ea-4ee5-bbd0-60d008969f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:03.032696 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211382f-39ea-4ee5-bbd0-60d008969f6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:03.032696 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211382f-39ea-4ee5-bbd0-60d008969f6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:44:03.032696 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211382f-39ea-4ee5-bbd0-60d008969f6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:03.032696 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211382f-39ea-4ee5-bbd0-60d008969f6c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:03.041636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1240/2480] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:44:03.046180 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252a379-62b5-40f6-88d7-50af41747b0a None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:03.055572 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252a379-62b5-40f6-88d7-50af41747b0a None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:03.060266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf136da-565b-47cc-96b1-f12d509e236c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:03.060875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf136da-565b-47cc-96b1-f12d509e236c admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:03.061186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf136da-565b-47cc-96b1-f12d509e236c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:03.061475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf136da-565b-47cc-96b1-f12d509e236c admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:03.062111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf136da-565b-47cc-96b1-f12d509e236c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:03.064068 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf136da-565b-47cc-96b1-f12d509e236c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:03.199999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1241/2481] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:44:03 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:44:03.204528 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211382f-39ea-4ee5-bbd0-60d008969f6c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:03.215273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211382f-39ea-4ee5-bbd0-60d008969f6c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:03.273253 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a23fe-cdc9-4d38-aa9e-850771b14fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:03.273851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a23fe-cdc9-4d38-aa9e-850771b14fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:03.274152 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a23fe-cdc9-4d38-aa9e-850771b14fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:03.274441 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a23fe-cdc9-4d38-aa9e-850771b14fe3 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:03.275052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a23fe-cdc9-4d38-aa9e-850771b14fe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:03.276975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a23fe-cdc9-4d38-aa9e-850771b14fe3 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:03.340742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1241/2482] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:44:03 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:44:03.344971 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf136da-565b-47cc-96b1-f12d509e236c admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:03.358958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf136da-565b-47cc-96b1-f12d509e236c admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:03.363836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bea4bb-e93c-46e7-9bcc-fe070b3cae6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:03.364438 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bea4bb-e93c-46e7-9bcc-fe070b3cae6f admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:03.364718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bea4bb-e93c-46e7-9bcc-fe070b3cae6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:03.364995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bea4bb-e93c-46e7-9bcc-fe070b3cae6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:03.365664 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bea4bb-e93c-46e7-9bcc-fe070b3cae6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:03.367267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bea4bb-e93c-46e7-9bcc-fe070b3cae6f admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:03.377337 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1242/2483] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:44:03.381353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a23fe-cdc9-4d38-aa9e-850771b14fe3 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:03.391269 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a23fe-cdc9-4d38-aa9e-850771b14fe3 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:03.396806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811929c-962f-4d02-9a61-15cb76942645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:03.397438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811929c-962f-4d02-9a61-15cb76942645 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:03.397721 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811929c-962f-4d02-9a61-15cb76942645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:03.397994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811929c-962f-4d02-9a61-15cb76942645 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:03.398635 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811929c-962f-4d02-9a61-15cb76942645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:03.400686 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811929c-962f-4d02-9a61-15cb76942645 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:03.411761 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1242/2484] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:44:03.415824 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bea4bb-e93c-46e7-9bcc-fe070b3cae6f admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:03.425667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bea4bb-e93c-46e7-9bcc-fe070b3cae6f admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:03.430300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72df03d-78fd-4b71-95e8-68f5cfa9d92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:03.430939 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72df03d-78fd-4b71-95e8-68f5cfa9d92d admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:03.431220 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72df03d-78fd-4b71-95e8-68f5cfa9d92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:03.431498 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72df03d-78fd-4b71-95e8-68f5cfa9d92d admin admin] PATH_INFO: `/v3/projects/1f4e750c130449f3b2c26eab08d7a89c/users/3c9ce59153a4420a8fa0b3025f70a808/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:03.445693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72df03d-78fd-4b71-95e8-68f5cfa9d92d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4e750c130449f3b2c26eab08d7a89c, user_id=3c9ce59153a4420a8fa0b3025f70a808, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:03.448339 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72df03d-78fd-4b71-95e8-68f5cfa9d92d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:03.507709 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:03.513883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1243/2485] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:03 2022] PUT /identity/v3/projects/1f4e750c130449f3b2c26eab08d7a89c/users/3c9ce59153a4420a8fa0b3025f70a808/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:03.526565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b90e2-02c1-4191-b76e-4f081503363c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:03.526857 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b90e2-02c1-4191-b76e-4f081503363c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:03.527134 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b90e2-02c1-4191-b76e-4f081503363c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:03.555302 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-de7b90e2-02c1-4191-b76e-4f081503363c None None] MFA Rules not processed for user `3c9ce59153a4420a8fa0b3025f70a808`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:03.556497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-de7b90e2-02c1-4191-b76e-4f081503363c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:03.623317 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1243/2486] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:03.628137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72df03d-78fd-4b71-95e8-68f5cfa9d92d admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:03.696273 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72df03d-78fd-4b71-95e8-68f5cfa9d92d admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:03.701500 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ec078-ee49-45c0-8f7d-5e4c9dbc8e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:03.702183 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ec078-ee49-45c0-8f7d-5e4c9dbc8e26 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:03.702479 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ec078-ee49-45c0-8f7d-5e4c9dbc8e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:03.702783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ec078-ee49-45c0-8f7d-5e4c9dbc8e26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:03.703481 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ec078-ee49-45c0-8f7d-5e4c9dbc8e26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:03.705629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ec078-ee49-45c0-8f7d-5e4c9dbc8e26 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:03.801566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1244/2487] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:44:03 2022] POST /identity/v3/projects => generated 371 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:44:03.805706 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b90e2-02c1-4191-b76e-4f081503363c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:03.817353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b90e2-02c1-4191-b76e-4f081503363c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:03.858071 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:03.874856 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b00d7-cd57-4474-90dc-756f41de2319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:03.875623 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b00d7-cd57-4474-90dc-756f41de2319 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:03.875901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b00d7-cd57-4474-90dc-756f41de2319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:03.876171 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b00d7-cd57-4474-90dc-756f41de2319 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:03.876867 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b00d7-cd57-4474-90dc-756f41de2319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:03.879502 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b00d7-cd57-4474-90dc-756f41de2319 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:04.005797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1244/2488] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:44:03 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:44:04.010004 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ec078-ee49-45c0-8f7d-5e4c9dbc8e26 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:04.023288 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ec078-ee49-45c0-8f7d-5e4c9dbc8e26 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:04.028787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf34332-efc7-4fcb-96d5-cc398b3af894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:04.029560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf34332-efc7-4fcb-96d5-cc398b3af894 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:04.029928 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf34332-efc7-4fcb-96d5-cc398b3af894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:04.030258 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf34332-efc7-4fcb-96d5-cc398b3af894 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:04.031003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf34332-efc7-4fcb-96d5-cc398b3af894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:04.037807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf34332-efc7-4fcb-96d5-cc398b3af894 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:04.052731 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1245/2489] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:44:04.057277 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b00d7-cd57-4474-90dc-756f41de2319 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:04.070043 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b00d7-cd57-4474-90dc-756f41de2319 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:04.075233 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33538fbb-54c4-40b3-bb65-705809045b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:04.075953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33538fbb-54c4-40b3-bb65-705809045b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:04.076294 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33538fbb-54c4-40b3-bb65-705809045b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:04.076570 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33538fbb-54c4-40b3-bb65-705809045b43 admin admin] PATH_INFO: `/v3/projects/56389c310a944dceb8aa7271f5237763/users/3b3f2b85e94e4e0d881358eb7ac4d647/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:04.093443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33538fbb-54c4-40b3-bb65-705809045b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56389c310a944dceb8aa7271f5237763, user_id=3b3f2b85e94e4e0d881358eb7ac4d647, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:04.096456 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33538fbb-54c4-40b3-bb65-705809045b43 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:04.199699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1245/2490] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:04 2022] PUT /identity/v3/projects/56389c310a944dceb8aa7271f5237763/users/3b3f2b85e94e4e0d881358eb7ac4d647/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:04.203733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf34332-efc7-4fcb-96d5-cc398b3af894 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:04.215250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf34332-efc7-4fcb-96d5-cc398b3af894 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:04.281045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699e6dc-531a-49b9-97ae-9628a897e87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:04.281045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699e6dc-531a-49b9-97ae-9628a897e87a admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:04.281045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699e6dc-531a-49b9-97ae-9628a897e87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:04.281404 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699e6dc-531a-49b9-97ae-9628a897e87a admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:04.281809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699e6dc-531a-49b9-97ae-9628a897e87a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:04.283425 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699e6dc-531a-49b9-97ae-9628a897e87a admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:04.294416 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1246/2491] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:44:04.299190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33538fbb-54c4-40b3-bb65-705809045b43 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:04.316950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33538fbb-54c4-40b3-bb65-705809045b43 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:04.319059 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd98f9-b29c-4ec3-8249-433c01892494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:04.319989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd98f9-b29c-4ec3-8249-433c01892494 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:04.320317 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd98f9-b29c-4ec3-8249-433c01892494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:04.320629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd98f9-b29c-4ec3-8249-433c01892494 admin admin] PATH_INFO: `/v3/projects/56389c310a944dceb8aa7271f5237763/users/3b3f2b85e94e4e0d881358eb7ac4d647/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:04.323750 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd98f9-b29c-4ec3-8249-433c01892494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56389c310a944dceb8aa7271f5237763, user_id=3b3f2b85e94e4e0d881358eb7ac4d647, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:04.327726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd98f9-b29c-4ec3-8249-433c01892494 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:04.342902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1246/2492] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:04 2022] PUT /identity/v3/projects/56389c310a944dceb8aa7271f5237763/users/3b3f2b85e94e4e0d881358eb7ac4d647/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:04.353972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd013d7b-9466-4076-a199-3ad89ddda8ba None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:04.354217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd013d7b-9466-4076-a199-3ad89ddda8ba None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:04.354569 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd013d7b-9466-4076-a199-3ad89ddda8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:04.386599 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-bd013d7b-9466-4076-a199-3ad89ddda8ba None None] MFA Rules not processed for user `3b3f2b85e94e4e0d881358eb7ac4d647`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:04.388249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-bd013d7b-9466-4076-a199-3ad89ddda8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:04.457362 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:04.468710 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1247/2493] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:04.482267 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd98f9-b29c-4ec3-8249-433c01892494 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:04.484146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-63fd98f9-b29c-4ec3-8249-433c01892494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:04.551779 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd98f9-b29c-4ec3-8249-433c01892494 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:04.553338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7a23d-7153-45ee-9385-a9bc8d941cda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:04.553948 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7a23d-7153-45ee-9385-a9bc8d941cda service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:04.554271 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7a23d-7153-45ee-9385-a9bc8d941cda service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:04.554494 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7a23d-7153-45ee-9385-a9bc8d941cda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:04.565845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7a23d-7153-45ee-9385-a9bc8d941cda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:04.567342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7a23d-7153-45ee-9385-a9bc8d941cda service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:04.577229 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1247/2494] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:44:05.179896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd013d7b-9466-4076-a199-3ad89ddda8ba None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:05.179896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-bd013d7b-9466-4076-a199-3ad89ddda8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:05.241653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd013d7b-9466-4076-a199-3ad89ddda8ba None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:05.247186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54220cca-d9b8-4b3a-a038-7ee165d73743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:05.247897 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54220cca-d9b8-4b3a-a038-7ee165d73743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:05.248258 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54220cca-d9b8-4b3a-a038-7ee165d73743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:05.248617 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54220cca-d9b8-4b3a-a038-7ee165d73743 admin admin] PATH_INFO: `/v3/users/498f068f5ed245e3aed418d31c9dd08f` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:05.249999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54220cca-d9b8-4b3a-a038-7ee165d73743 admin admin] RBAC: Authorizing `identity:delete_user(user_id=498f068f5ed245e3aed418d31c9dd08f)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:05.252346 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54220cca-d9b8-4b3a-a038-7ee165d73743 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:05.585188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-54220cca-d9b8-4b3a-a038-7ee165d73743 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '498f068f5ed245e3aed418d31c9dd08f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:05.701594 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-54220cca-d9b8-4b3a-a038-7ee165d73743 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '498f068f5ed245e3aed418d31c9dd08f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:05.702318 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-54220cca-d9b8-4b3a-a038-7ee165d73743 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '498f068f5ed245e3aed418d31c9dd08f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:05.703004 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-54220cca-d9b8-4b3a-a038-7ee165d73743 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '498f068f5ed245e3aed418d31c9dd08f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:05.711710 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-54220cca-d9b8-4b3a-a038-7ee165d73743 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '498f068f5ed245e3aed418d31c9dd08f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:05.731511 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1248/2495] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:44:05 2022] DELETE /identity/v3/users/498f068f5ed245e3aed418d31c9dd08f => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:05.736435 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981c68e-5353-4da5-bee7-f2f0c98d0b3d None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:05.736933 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981c68e-5353-4da5-bee7-f2f0c98d0b3d None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:05.737202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981c68e-5353-4da5-bee7-f2f0c98d0b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:05.759052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-7981c68e-5353-4da5-bee7-f2f0c98d0b3d None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:05.760477 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7981c68e-5353-4da5-bee7-f2f0c98d0b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:05.814917 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54220cca-d9b8-4b3a-a038-7ee165d73743 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:05.816967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-54220cca-d9b8-4b3a-a038-7ee165d73743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:05.833789 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1248/2496] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:05.844013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981c68e-5353-4da5-bee7-f2f0c98d0b3d None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:05.845374 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7981c68e-5353-4da5-bee7-f2f0c98d0b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:05.887402 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:05.893549 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54220cca-d9b8-4b3a-a038-7ee165d73743 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:05.899670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffb6ee-4869-4707-b8e6-9bfabd3f594f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:05.900393 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffb6ee-4869-4707-b8e6-9bfabd3f594f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:05.900683 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffb6ee-4869-4707-b8e6-9bfabd3f594f admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:05.900991 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffb6ee-4869-4707-b8e6-9bfabd3f594f admin admin] PATH_INFO: `/v3/users/3c9ce59153a4420a8fa0b3025f70a808` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:05.902341 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffb6ee-4869-4707-b8e6-9bfabd3f594f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c9ce59153a4420a8fa0b3025f70a808)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:05.904519 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffb6ee-4869-4707-b8e6-9bfabd3f594f admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:05.906087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981c68e-5353-4da5-bee7-f2f0c98d0b3d None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:05.909858 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31ecce-29d2-4c97-8f82-7bc77ddce438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:05.910485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31ecce-29d2-4c97-8f82-7bc77ddce438 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:05.910861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31ecce-29d2-4c97-8f82-7bc77ddce438 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:05.911139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31ecce-29d2-4c97-8f82-7bc77ddce438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:05.922756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:05.923840 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31ecce-29d2-4c97-8f82-7bc77ddce438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:05.925406 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31ecce-29d2-4c97-8f82-7bc77ddce438 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:05.935655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1249/2497] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:44:06.164051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-32ffb6ee-4869-4707-b8e6-9bfabd3f594f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c9ce59153a4420a8fa0b3025f70a808'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:06.245996 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-32ffb6ee-4869-4707-b8e6-9bfabd3f594f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c9ce59153a4420a8fa0b3025f70a808'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:06.247028 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-32ffb6ee-4869-4707-b8e6-9bfabd3f594f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c9ce59153a4420a8fa0b3025f70a808'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:06.247796 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-32ffb6ee-4869-4707-b8e6-9bfabd3f594f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c9ce59153a4420a8fa0b3025f70a808'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:06.259587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-32ffb6ee-4869-4707-b8e6-9bfabd3f594f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c9ce59153a4420a8fa0b3025f70a808'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:06.306969 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1249/2498] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:44:05 2022] DELETE /identity/v3/users/3c9ce59153a4420a8fa0b3025f70a808 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:06.311914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a7205-1aaf-4bbd-9670-91bac34080d7 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:06.312191 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a7205-1aaf-4bbd-9670-91bac34080d7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:06.312469 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a7205-1aaf-4bbd-9670-91bac34080d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:06.333082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-141a7205-1aaf-4bbd-9670-91bac34080d7 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:06.334409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-141a7205-1aaf-4bbd-9670-91bac34080d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:06.397437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1250/2499] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:06.407672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffb6ee-4869-4707-b8e6-9bfabd3f594f admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:06.409278 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-32ffb6ee-4869-4707-b8e6-9bfabd3f594f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:06.422513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a7205-1aaf-4bbd-9670-91bac34080d7 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:06.423895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-141a7205-1aaf-4bbd-9670-91bac34080d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:06.438980 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a7205-1aaf-4bbd-9670-91bac34080d7 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:06.444747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffb844-aaaa-49e8-8ba1-fe6b03ce0a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:06.445573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffb844-aaaa-49e8-8ba1-fe6b03ce0a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:06.445865 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffb844-aaaa-49e8-8ba1-fe6b03ce0a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:06.446137 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffb844-aaaa-49e8-8ba1-fe6b03ce0a8d admin admin] PATH_INFO: `/v3/projects/f96ae4bdab0349ed859c7ae8aa59ac6b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:06.447442 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abffb844-aaaa-49e8-8ba1-fe6b03ce0a8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96ae4bdab0349ed859c7ae8aa59ac6b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:06.450170 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abffb844-aaaa-49e8-8ba1-fe6b03ce0a8d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:06.489893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffb6ee-4869-4707-b8e6-9bfabd3f594f admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:06.495543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efddca2c-2149-420e-b4c9-5b3be3dbe758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:06.495543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efddca2c-2149-420e-b4c9-5b3be3dbe758 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:06.495543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efddca2c-2149-420e-b4c9-5b3be3dbe758 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:06.495543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efddca2c-2149-420e-b4c9-5b3be3dbe758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:06.506143 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efddca2c-2149-420e-b4c9-5b3be3dbe758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:06.507616 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efddca2c-2149-420e-b4c9-5b3be3dbe758 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:06.518648 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1250/2500] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:44:06.561254 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-abffb844-aaaa-49e8-8ba1-fe6b03ce0a8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96ae4bdab0349ed859c7ae8aa59ac6b', 'request_id': 'req-abffb844-aaaa-49e8-8ba1-fe6b03ce0a8d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:06.563649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-abffb844-aaaa-49e8-8ba1-fe6b03ce0a8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96ae4bdab0349ed859c7ae8aa59ac6b', 'request_id': 'req-abffb844-aaaa-49e8-8ba1-fe6b03ce0a8d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:06.572506 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-abffb844-aaaa-49e8-8ba1-fe6b03ce0a8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96ae4bdab0349ed859c7ae8aa59ac6b', 'request_id': 'req-abffb844-aaaa-49e8-8ba1-fe6b03ce0a8d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:06.643628 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-abffb844-aaaa-49e8-8ba1-fe6b03ce0a8d admin admin] The token cache is being invalidate because project f96ae4bdab0349ed859c7ae8aa59ac6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:44:06.643919 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-abffb844-aaaa-49e8-8ba1-fe6b03ce0a8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:06.645683 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-abffb844-aaaa-49e8-8ba1-fe6b03ce0a8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:06.650531 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1251/2501] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:44:06 2022] DELETE /identity/v3/projects/f96ae4bdab0349ed859c7ae8aa59ac6b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:06.651863 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efddca2c-2149-420e-b4c9-5b3be3dbe758 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:06.662343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efddca2c-2149-420e-b4c9-5b3be3dbe758 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:06.728165 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22faca-8acb-423e-acb5-a5b5fa3fcf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:06.736266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22faca-8acb-423e-acb5-a5b5fa3fcf64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:06.736266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22faca-8acb-423e-acb5-a5b5fa3fcf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:06.736266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22faca-8acb-423e-acb5-a5b5fa3fcf64 admin admin] PATH_INFO: `/v3/users/e0255aa04e7e42539a357c57187b1eca` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:06.736266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22faca-8acb-423e-acb5-a5b5fa3fcf64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0255aa04e7e42539a357c57187b1eca)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:06.736266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22faca-8acb-423e-acb5-a5b5fa3fcf64 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:06.983497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffb844-aaaa-49e8-8ba1-fe6b03ce0a8d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:06.990335 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:06.990588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-abffb844-aaaa-49e8-8ba1-fe6b03ce0a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:07.007709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffb844-aaaa-49e8-8ba1-fe6b03ce0a8d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:07.010652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7e22faca-8acb-423e-acb5-a5b5fa3fcf64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0255aa04e7e42539a357c57187b1eca'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:07.015437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564779d5-5ade-459b-b8c2-503416de53aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:07.016153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564779d5-5ade-459b-b8c2-503416de53aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:07.016459 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564779d5-5ade-459b-b8c2-503416de53aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:07.016719 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564779d5-5ade-459b-b8c2-503416de53aa admin admin] PATH_INFO: `/v3/projects/1f4e750c130449f3b2c26eab08d7a89c` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:07.020147 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564779d5-5ade-459b-b8c2-503416de53aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f4e750c130449f3b2c26eab08d7a89c)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:07.023552 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564779d5-5ade-459b-b8c2-503416de53aa admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:07.198694 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7e22faca-8acb-423e-acb5-a5b5fa3fcf64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0255aa04e7e42539a357c57187b1eca'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:07.199334 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7e22faca-8acb-423e-acb5-a5b5fa3fcf64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0255aa04e7e42539a357c57187b1eca'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:07.199973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7e22faca-8acb-423e-acb5-a5b5fa3fcf64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0255aa04e7e42539a357c57187b1eca'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:07.209508 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7e22faca-8acb-423e-acb5-a5b5fa3fcf64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0255aa04e7e42539a357c57187b1eca'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:07.219356 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-564779d5-5ade-459b-b8c2-503416de53aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f4e750c130449f3b2c26eab08d7a89c', 'request_id': 'req-564779d5-5ade-459b-b8c2-503416de53aa'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:07.226165 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-564779d5-5ade-459b-b8c2-503416de53aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f4e750c130449f3b2c26eab08d7a89c', 'request_id': 'req-564779d5-5ade-459b-b8c2-503416de53aa'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:07.227415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:07.228319 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1251/2502] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:44:06 2022] DELETE /identity/v3/users/e0255aa04e7e42539a357c57187b1eca => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:07.236531 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-564779d5-5ade-459b-b8c2-503416de53aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f4e750c130449f3b2c26eab08d7a89c', 'request_id': 'req-564779d5-5ade-459b-b8c2-503416de53aa'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:07.426354 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-564779d5-5ade-459b-b8c2-503416de53aa admin admin] The token cache is being invalidate because project 1f4e750c130449f3b2c26eab08d7a89c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:44:07.426876 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-564779d5-5ade-459b-b8c2-503416de53aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:07.427642 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-564779d5-5ade-459b-b8c2-503416de53aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:07.429584 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1252/2503] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:44:06 2022] DELETE /identity/v3/projects/1f4e750c130449f3b2c26eab08d7a89c => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:07.437513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22faca-8acb-423e-acb5-a5b5fa3fcf64 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:07.439093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-7e22faca-8acb-423e-acb5-a5b5fa3fcf64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:07.521384 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22faca-8acb-423e-acb5-a5b5fa3fcf64 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:07.529881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42723094-1161-4379-9aee-c58ac0c08ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:07.531347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42723094-1161-4379-9aee-c58ac0c08ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:07.532546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42723094-1161-4379-9aee-c58ac0c08ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:07.532936 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42723094-1161-4379-9aee-c58ac0c08ade admin admin] PATH_INFO: `/v3/users/3b3f2b85e94e4e0d881358eb7ac4d647` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:07.539948 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42723094-1161-4379-9aee-c58ac0c08ade admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b3f2b85e94e4e0d881358eb7ac4d647)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:07.539948 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42723094-1161-4379-9aee-c58ac0c08ade admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:07.832884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564779d5-5ade-459b-b8c2-503416de53aa admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:07.835160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-564779d5-5ade-459b-b8c2-503416de53aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:07.852112 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-42723094-1161-4379-9aee-c58ac0c08ade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b3f2b85e94e4e0d881358eb7ac4d647'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:07.932545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564779d5-5ade-459b-b8c2-503416de53aa admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:07.939147 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ceb8d-0b2b-4189-ab27-c988c0284476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:07.939993 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ceb8d-0b2b-4189-ab27-c988c0284476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:07.940329 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ceb8d-0b2b-4189-ab27-c988c0284476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:07.940738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ceb8d-0b2b-4189-ab27-c988c0284476 admin admin] PATH_INFO: `/v3/projects/bd61355891c74819b1194dd07026a9b2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:07.943017 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ceb8d-0b2b-4189-ab27-c988c0284476 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd61355891c74819b1194dd07026a9b2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:07.945858 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ceb8d-0b2b-4189-ab27-c988c0284476 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:07.946407 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-42723094-1161-4379-9aee-c58ac0c08ade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b3f2b85e94e4e0d881358eb7ac4d647'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:07.947350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-42723094-1161-4379-9aee-c58ac0c08ade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b3f2b85e94e4e0d881358eb7ac4d647'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:07.948107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-42723094-1161-4379-9aee-c58ac0c08ade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b3f2b85e94e4e0d881358eb7ac4d647'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:07.956159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-42723094-1161-4379-9aee-c58ac0c08ade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b3f2b85e94e4e0d881358eb7ac4d647'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:07.977174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1252/2504] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:44:07 2022] DELETE /identity/v3/users/3b3f2b85e94e4e0d881358eb7ac4d647 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:08.084109 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-641ceb8d-0b2b-4189-ab27-c988c0284476 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd61355891c74819b1194dd07026a9b2', 'request_id': 'req-641ceb8d-0b2b-4189-ab27-c988c0284476'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:08.091931 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-641ceb8d-0b2b-4189-ab27-c988c0284476 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd61355891c74819b1194dd07026a9b2', 'request_id': 'req-641ceb8d-0b2b-4189-ab27-c988c0284476'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:08.098541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-641ceb8d-0b2b-4189-ab27-c988c0284476 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd61355891c74819b1194dd07026a9b2', 'request_id': 'req-641ceb8d-0b2b-4189-ab27-c988c0284476'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:08.211909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-641ceb8d-0b2b-4189-ab27-c988c0284476 admin admin] The token cache is being invalidate because project bd61355891c74819b1194dd07026a9b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:44:08.212194 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-641ceb8d-0b2b-4189-ab27-c988c0284476 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:08.212842 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-641ceb8d-0b2b-4189-ab27-c988c0284476 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:08.218555 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1253/2505] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:44:07 2022] DELETE /identity/v3/projects/bd61355891c74819b1194dd07026a9b2 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:08.220187 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c3951-abb0-4f10-aa31-14236a9bab34 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:08.220474 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c3951-abb0-4f10-aa31-14236a9bab34 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:08.220751 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c3951-abb0-4f10-aa31-14236a9bab34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:08.241140 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-651c3951-abb0-4f10-aa31-14236a9bab34 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:08.242303 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-651c3951-abb0-4f10-aa31-14236a9bab34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:08.331482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1253/2506] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:08.341200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb6108-7132-4ff7-b0dd-ded1e7234f9f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:08.341480 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb6108-7132-4ff7-b0dd-ded1e7234f9f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:08.341757 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb6108-7132-4ff7-b0dd-ded1e7234f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:08.374821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-80bb6108-7132-4ff7-b0dd-ded1e7234f9f None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:08.376121 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-80bb6108-7132-4ff7-b0dd-ded1e7234f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:08.391711 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1254/2507] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:08.392999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c3951-abb0-4f10-aa31-14236a9bab34 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:08.404904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c3951-abb0-4f10-aa31-14236a9bab34 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:08.421893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb6d90-d6ae-4a68-ba88-e74ba844c69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:08.422909 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb6d90-d6ae-4a68-ba88-e74ba844c69d admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:08.424425 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb6d90-d6ae-4a68-ba88-e74ba844c69d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:08.424701 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb6d90-d6ae-4a68-ba88-e74ba844c69d admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:08.426299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb6d90-d6ae-4a68-ba88-e74ba844c69d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:44:08.430838 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb6d90-d6ae-4a68-ba88-e74ba844c69d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:08.430838 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb6d90-d6ae-4a68-ba88-e74ba844c69d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:08.445109 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1254/2508] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:44:08.448941 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb6108-7132-4ff7-b0dd-ded1e7234f9f None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:08.461006 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb6108-7132-4ff7-b0dd-ded1e7234f9f None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:08.474175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb93bee-7a46-4f71-9f7d-a20abeb2fed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:08.474954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb93bee-7a46-4f71-9f7d-a20abeb2fed8 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:08.475274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb93bee-7a46-4f71-9f7d-a20abeb2fed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:08.475596 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb93bee-7a46-4f71-9f7d-a20abeb2fed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:08.476399 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb93bee-7a46-4f71-9f7d-a20abeb2fed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:08.478926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb93bee-7a46-4f71-9f7d-a20abeb2fed8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:08.523275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb6d90-d6ae-4a68-ba88-e74ba844c69d admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:08.528545 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c3bb6d90-d6ae-4a68-ba88-e74ba844c69d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:08.540434 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb6d90-d6ae-4a68-ba88-e74ba844c69d admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:08.547861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57abc0-6f53-4449-8989-fff120f137f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:08.549080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57abc0-6f53-4449-8989-fff120f137f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:08.549700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57abc0-6f53-4449-8989-fff120f137f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:08.550281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57abc0-6f53-4449-8989-fff120f137f3 admin admin] PATH_INFO: `/v3/projects/56389c310a944dceb8aa7271f5237763` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:08.552448 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57abc0-6f53-4449-8989-fff120f137f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56389c310a944dceb8aa7271f5237763)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:08.554584 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57abc0-6f53-4449-8989-fff120f137f3 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:08.616583 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:08.650233 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1255/2509] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:44:08 2022] POST /identity/v3/projects => generated 389 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:44:08.651864 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb93bee-7a46-4f71-9f7d-a20abeb2fed8 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:08.723659 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb93bee-7a46-4f71-9f7d-a20abeb2fed8 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:08.723837 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-fd57abc0-6f53-4449-8989-fff120f137f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56389c310a944dceb8aa7271f5237763', 'request_id': 'req-fd57abc0-6f53-4449-8989-fff120f137f3'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:08.732649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-fd57abc0-6f53-4449-8989-fff120f137f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56389c310a944dceb8aa7271f5237763', 'request_id': 'req-fd57abc0-6f53-4449-8989-fff120f137f3'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:08.748965 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-fd57abc0-6f53-4449-8989-fff120f137f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56389c310a944dceb8aa7271f5237763', 'request_id': 'req-fd57abc0-6f53-4449-8989-fff120f137f3'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:08.791080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412ebaf-601e-4ea1-8357-bafcba3203b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:08.791701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412ebaf-601e-4ea1-8357-bafcba3203b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:08.791981 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412ebaf-601e-4ea1-8357-bafcba3203b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:08.792258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412ebaf-601e-4ea1-8357-bafcba3203b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:08.793036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412ebaf-601e-4ea1-8357-bafcba3203b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:08.795062 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412ebaf-601e-4ea1-8357-bafcba3203b0 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:08.898478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-fd57abc0-6f53-4449-8989-fff120f137f3 admin admin] The token cache is being invalidate because project 56389c310a944dceb8aa7271f5237763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:44:08.898847 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-fd57abc0-6f53-4449-8989-fff120f137f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:08.899571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-fd57abc0-6f53-4449-8989-fff120f137f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:08.901884 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1255/2510] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:44:08 2022] DELETE /identity/v3/projects/56389c310a944dceb8aa7271f5237763 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:08.906782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a02b44-9da7-4273-aa54-5a2798e3032b None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:08.907199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a02b44-9da7-4273-aa54-5a2798e3032b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:08.907486 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a02b44-9da7-4273-aa54-5a2798e3032b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:08.940406 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1256/2511] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:44:08 2022] POST /identity/v3/users => generated 295 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:44:08.945967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412ebaf-601e-4ea1-8357-bafcba3203b0 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:08.948028 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4412ebaf-601e-4ea1-8357-bafcba3203b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:08.952667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-84a02b44-9da7-4273-aa54-5a2798e3032b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:08.954487 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-84a02b44-9da7-4273-aa54-5a2798e3032b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:08.966673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412ebaf-601e-4ea1-8357-bafcba3203b0 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:08.968890 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1256/2512] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:08.979544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68e7d6-eeb8-4c78-9ebb-47e69ef373a6 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:08.980415 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68e7d6-eeb8-4c78-9ebb-47e69ef373a6 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:08.981015 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68e7d6-eeb8-4c78-9ebb-47e69ef373a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:08.983464 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca32a4-7470-49ea-9747-84e177f6c73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:08.984428 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca32a4-7470-49ea-9747-84e177f6c73a admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:08.984716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca32a4-7470-49ea-9747-84e177f6c73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:08.984985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca32a4-7470-49ea-9747-84e177f6c73a admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:08.985686 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca32a4-7470-49ea-9747-84e177f6c73a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:08.987262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca32a4-7470-49ea-9747-84e177f6c73a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:09.002446 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1257/2513] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:44:09.013763 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-0a68e7d6-eeb8-4c78-9ebb-47e69ef373a6 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:09.018948 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-0a68e7d6-eeb8-4c78-9ebb-47e69ef373a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:09.022978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca32a4-7470-49ea-9747-84e177f6c73a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:09.025951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1257/2514] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:09.030698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68e7d6-eeb8-4c78-9ebb-47e69ef373a6 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:09.033766 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca32a4-7470-49ea-9747-84e177f6c73a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:09.039285 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e35dea-18ff-4452-9e85-01c125096f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:09.040174 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e35dea-18ff-4452-9e85-01c125096f8e admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:09.040528 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e35dea-18ff-4452-9e85-01c125096f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:09.040873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e35dea-18ff-4452-9e85-01c125096f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:09.041503 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68e7d6-eeb8-4c78-9ebb-47e69ef373a6 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:09.041699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e35dea-18ff-4452-9e85-01c125096f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:09.043678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e35dea-18ff-4452-9e85-01c125096f8e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:09.047883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b0b1a-6feb-4414-9d04-f622327e0a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:09.047883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b0b1a-6feb-4414-9d04-f622327e0a47 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:09.049158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b0b1a-6feb-4414-9d04-f622327e0a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:09.049158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b0b1a-6feb-4414-9d04-f622327e0a47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:09.049441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b0b1a-6feb-4414-9d04-f622327e0a47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:44:09.049832 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b0b1a-6feb-4414-9d04-f622327e0a47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:09.051801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b0b1a-6feb-4414-9d04-f622327e0a47 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:09.055945 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1258/2515] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:44:09.060315 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e35dea-18ff-4452-9e85-01c125096f8e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:09.066904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1258/2516] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:44:09.071372 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b0b1a-6feb-4414-9d04-f622327e0a47 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:09.077812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e35dea-18ff-4452-9e85-01c125096f8e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:09.077812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3268f-d5fc-4636-ad40-89890d37f9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:09.078446 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3268f-d5fc-4636-ad40-89890d37f9be admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:09.078718 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3268f-d5fc-4636-ad40-89890d37f9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:09.078993 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3268f-d5fc-4636-ad40-89890d37f9be admin admin] PATH_INFO: `/v3/projects/c507ad8808854f54b328a9a44f62ab7d/users/c99c5839e57f494f8fcc019dd8e7f4c3/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:09.081377 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b0b1a-6feb-4414-9d04-f622327e0a47 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:09.086665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b951186-dc0c-47e8-b777-ab2dc2001313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:09.087264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b951186-dc0c-47e8-b777-ab2dc2001313 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:09.087583 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b951186-dc0c-47e8-b777-ab2dc2001313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:09.087850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b951186-dc0c-47e8-b777-ab2dc2001313 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:09.088476 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b951186-dc0c-47e8-b777-ab2dc2001313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:09.090408 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b951186-dc0c-47e8-b777-ab2dc2001313 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:09.097799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3268f-d5fc-4636-ad40-89890d37f9be admin admin] RBAC: Authorizing `identity:create_grant(project_id=c507ad8808854f54b328a9a44f62ab7d, user_id=c99c5839e57f494f8fcc019dd8e7f4c3, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:09.107293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:09.107293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3268f-d5fc-4636-ad40-89890d37f9be admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:09.193183 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1259/2517] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:09 2022] PUT /identity/v3/projects/c507ad8808854f54b328a9a44f62ab7d/users/c99c5839e57f494f8fcc019dd8e7f4c3/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:09.195510 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1259/2518] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:44:09 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:44:09.199861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3268f-d5fc-4636-ad40-89890d37f9be admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:09.204524 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa4d2d-641e-47d3-9992-e2aca296e93e None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:09.206297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa4d2d-641e-47d3-9992-e2aca296e93e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:09.206297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa4d2d-641e-47d3-9992-e2aca296e93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:09.234750 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-37fa4d2d-641e-47d3-9992-e2aca296e93e None None] MFA Rules not processed for user `c99c5839e57f494f8fcc019dd8e7f4c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:09.236003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-37fa4d2d-641e-47d3-9992-e2aca296e93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:09.274084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3268f-d5fc-4636-ad40-89890d37f9be admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:09.314226 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1260/2519] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:09.320032 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa4d2d-641e-47d3-9992-e2aca296e93e None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:09.332697 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906423f-3b80-4ae3-a209-1b2cb192e02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:09.333489 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906423f-3b80-4ae3-a209-1b2cb192e02a admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:09.333796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906423f-3b80-4ae3-a209-1b2cb192e02a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:09.334111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906423f-3b80-4ae3-a209-1b2cb192e02a admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:09.334809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906423f-3b80-4ae3-a209-1b2cb192e02a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:09.337006 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906423f-3b80-4ae3-a209-1b2cb192e02a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:09.393406 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa4d2d-641e-47d3-9992-e2aca296e93e None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:09.398444 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce91e4e-d0a1-47f4-b571-d6ec7f1ba32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:09.399045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce91e4e-d0a1-47f4-b571-d6ec7f1ba32c admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:09.399326 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce91e4e-d0a1-47f4-b571-d6ec7f1ba32c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:09.399605 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce91e4e-d0a1-47f4-b571-d6ec7f1ba32c admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:09.400235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce91e4e-d0a1-47f4-b571-d6ec7f1ba32c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:09.402195 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce91e4e-d0a1-47f4-b571-d6ec7f1ba32c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:09.432195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1260/2520] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:44:09 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:44:09.437295 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906423f-3b80-4ae3-a209-1b2cb192e02a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:09.447253 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906423f-3b80-4ae3-a209-1b2cb192e02a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:09.451909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504ca4e-1b81-46f9-8a1b-ea728b9748c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:09.452472 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504ca4e-1b81-46f9-8a1b-ea728b9748c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:09.452749 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504ca4e-1b81-46f9-8a1b-ea728b9748c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:09.453022 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504ca4e-1b81-46f9-8a1b-ea728b9748c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:09.453685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9504ca4e-1b81-46f9-8a1b-ea728b9748c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:09.455262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9504ca4e-1b81-46f9-8a1b-ea728b9748c9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:09.465228 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1261/2521] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:44:09.472935 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504ca4e-1b81-46f9-8a1b-ea728b9748c9 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:09.482259 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504ca4e-1b81-46f9-8a1b-ea728b9748c9 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:09.487026 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511a3ba-475b-4679-817d-9ae0007b3a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:09.487663 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511a3ba-475b-4679-817d-9ae0007b3a2d admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:09.487937 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511a3ba-475b-4679-817d-9ae0007b3a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:09.488798 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511a3ba-475b-4679-817d-9ae0007b3a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:09.489467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511a3ba-475b-4679-817d-9ae0007b3a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:09.491014 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511a3ba-475b-4679-817d-9ae0007b3a2d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:09.501412 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1262/2522] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:44:09.505394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511a3ba-475b-4679-817d-9ae0007b3a2d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:09.514767 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511a3ba-475b-4679-817d-9ae0007b3a2d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:09.519706 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e309419-a92b-4fce-b229-95eaf42b554a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:09.520334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e309419-a92b-4fce-b229-95eaf42b554a admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:09.520613 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e309419-a92b-4fce-b229-95eaf42b554a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:09.522220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e309419-a92b-4fce-b229-95eaf42b554a admin admin] PATH_INFO: `/v3/projects/aacbec8cbf9f4cab8157ec36571adc96/users/d6f8d7e0d2d6451090c20dae6543e59d/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:09.529741 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1261/2523] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:44:09 2022] POST /identity/v3/projects => generated 385 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:44:09.530665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce91e4e-d0a1-47f4-b571-d6ec7f1ba32c admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:09.537685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e309419-a92b-4fce-b229-95eaf42b554a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aacbec8cbf9f4cab8157ec36571adc96, user_id=d6f8d7e0d2d6451090c20dae6543e59d, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:09.540317 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e309419-a92b-4fce-b229-95eaf42b554a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:09.600770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce91e4e-d0a1-47f4-b571-d6ec7f1ba32c admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:09.606264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226335f-c711-4715-a94a-658a20bd1505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:09.607032 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226335f-c711-4715-a94a-658a20bd1505 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:09.607489 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226335f-c711-4715-a94a-658a20bd1505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:09.607976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226335f-c711-4715-a94a-658a20bd1505 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:09.608905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226335f-c711-4715-a94a-658a20bd1505 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:09.611424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226335f-c711-4715-a94a-658a20bd1505 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:09.621652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1263/2524] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:09 2022] PUT /identity/v3/projects/aacbec8cbf9f4cab8157ec36571adc96/users/d6f8d7e0d2d6451090c20dae6543e59d/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:09.633258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c3c4a-cdf6-489e-8136-66d9d6694635 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:09.636207 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c3c4a-cdf6-489e-8136-66d9d6694635 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:09.636207 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c3c4a-cdf6-489e-8136-66d9d6694635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:09.664559 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f26c3c4a-cdf6-489e-8136-66d9d6694635 None None] MFA Rules not processed for user `d6f8d7e0d2d6451090c20dae6543e59d`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:09.666003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f26c3c4a-cdf6-489e-8136-66d9d6694635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:09.732414 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1264/2525] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:09.737543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c3c4a-cdf6-489e-8136-66d9d6694635 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:09.777480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1262/2526] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:44:09 2022] POST /identity/v3/users => generated 299 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:44:09.781524 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226335f-c711-4715-a94a-658a20bd1505 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:09.791905 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226335f-c711-4715-a94a-658a20bd1505 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:09.802483 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c3c4a-cdf6-489e-8136-66d9d6694635 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:09.807398 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd592c45-bbdc-42e7-a176-e90935d42e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:09.808003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd592c45-bbdc-42e7-a176-e90935d42e02 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:09.808996 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd592c45-bbdc-42e7-a176-e90935d42e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:09.809389 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd592c45-bbdc-42e7-a176-e90935d42e02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:09.822262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd592c45-bbdc-42e7-a176-e90935d42e02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:09.822262 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd592c45-bbdc-42e7-a176-e90935d42e02 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:09.865031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bf7a6-52cf-49b6-95b5-a0add7256e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:09.867140 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bf7a6-52cf-49b6-95b5-a0add7256e41 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:09.867140 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bf7a6-52cf-49b6-95b5-a0add7256e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:09.867140 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bf7a6-52cf-49b6-95b5-a0add7256e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:09.867140 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84bf7a6-52cf-49b6-95b5-a0add7256e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:09.868693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84bf7a6-52cf-49b6-95b5-a0add7256e41 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:09.879284 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1263/2527] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:44:09.883498 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bf7a6-52cf-49b6-95b5-a0add7256e41 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:09.895766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bf7a6-52cf-49b6-95b5-a0add7256e41 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:09.901325 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20b05d-a64d-48b1-91c1-abdcb4f40bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:09.902044 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20b05d-a64d-48b1-91c1-abdcb4f40bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:09.902356 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20b05d-a64d-48b1-91c1-abdcb4f40bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:09.902695 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20b05d-a64d-48b1-91c1-abdcb4f40bc4 admin admin] PATH_INFO: `/v3/projects/277e958eb60840ac800db215a2a107f0/users/6727a005e30741c08aa0ac7e0e94708c/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:09.920527 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20b05d-a64d-48b1-91c1-abdcb4f40bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=277e958eb60840ac800db215a2a107f0, user_id=6727a005e30741c08aa0ac7e0e94708c, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:09.923723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20b05d-a64d-48b1-91c1-abdcb4f40bc4 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:09.948898 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1265/2528] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:44:09 2022] POST /identity/v3/projects => generated 377 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:44:09.953395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd592c45-bbdc-42e7-a176-e90935d42e02 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:10.004442 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1264/2529] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:09 2022] PUT /identity/v3/projects/277e958eb60840ac800db215a2a107f0/users/6727a005e30741c08aa0ac7e0e94708c/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:10.008895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20b05d-a64d-48b1-91c1-abdcb4f40bc4 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:10.026904 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd592c45-bbdc-42e7-a176-e90935d42e02 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:10.106630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20b05d-a64d-48b1-91c1-abdcb4f40bc4 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:10.108054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848569ce-8ea7-4a6c-a0c0-5c2f82681a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:10.108746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848569ce-8ea7-4a6c-a0c0-5c2f82681a7f admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:10.109111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848569ce-8ea7-4a6c-a0c0-5c2f82681a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:10.109464 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848569ce-8ea7-4a6c-a0c0-5c2f82681a7f admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:10.110367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848569ce-8ea7-4a6c-a0c0-5c2f82681a7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:10.112838 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848569ce-8ea7-4a6c-a0c0-5c2f82681a7f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:10.112997 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa238176-40b7-4100-b226-b0ad79030852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:10.113658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa238176-40b7-4100-b226-b0ad79030852 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:10.114016 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa238176-40b7-4100-b226-b0ad79030852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:10.114371 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa238176-40b7-4100-b226-b0ad79030852 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:10.115131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa238176-40b7-4100-b226-b0ad79030852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:10.117664 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa238176-40b7-4100-b226-b0ad79030852 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:10.132872 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1265/2530] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:44:10.137305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa238176-40b7-4100-b226-b0ad79030852 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:10.150154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa238176-40b7-4100-b226-b0ad79030852 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:10.155864 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b98f7-c3c7-4b84-960c-fffeec5c6a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:10.156604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b98f7-c3c7-4b84-960c-fffeec5c6a64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:10.156941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b98f7-c3c7-4b84-960c-fffeec5c6a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:10.157329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b98f7-c3c7-4b84-960c-fffeec5c6a64 admin admin] PATH_INFO: `/v3/projects/277e958eb60840ac800db215a2a107f0/users/6727a005e30741c08aa0ac7e0e94708c/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:10.159649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b98f7-c3c7-4b84-960c-fffeec5c6a64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=277e958eb60840ac800db215a2a107f0, user_id=6727a005e30741c08aa0ac7e0e94708c, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:10.163131 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b98f7-c3c7-4b84-960c-fffeec5c6a64 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:10.176237 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1266/2531] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:10 2022] PUT /identity/v3/projects/277e958eb60840ac800db215a2a107f0/users/6727a005e30741c08aa0ac7e0e94708c/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:10.191158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856f81b-2ecb-4048-b45f-ff1e96a3a908 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:10.191158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856f81b-2ecb-4048-b45f-ff1e96a3a908 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:10.191158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856f81b-2ecb-4048-b45f-ff1e96a3a908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:10.220288 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-4856f81b-2ecb-4048-b45f-ff1e96a3a908 None None] MFA Rules not processed for user `6727a005e30741c08aa0ac7e0e94708c`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:10.221614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4856f81b-2ecb-4048-b45f-ff1e96a3a908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:10.247176 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1266/2532] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:44:09 2022] POST /identity/v3/users => generated 295 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:44:10.251650 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848569ce-8ea7-4a6c-a0c0-5c2f82681a7f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:10.263159 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848569ce-8ea7-4a6c-a0c0-5c2f82681a7f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:10.310952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1267/2533] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:10.319547 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856f81b-2ecb-4048-b45f-ff1e96a3a908 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:10.321167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4856f81b-2ecb-4048-b45f-ff1e96a3a908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:10.342603 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f4563-7522-4380-b23e-bc067717c93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:10.343242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f4563-7522-4380-b23e-bc067717c93a admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:10.343525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f4563-7522-4380-b23e-bc067717c93a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:10.344211 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f4563-7522-4380-b23e-bc067717c93a admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:10.345160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f4563-7522-4380-b23e-bc067717c93a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:10.347139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f4563-7522-4380-b23e-bc067717c93a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:10.374261 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1267/2534] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:44:10.381504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f4563-7522-4380-b23e-bc067717c93a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:10.395715 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f4563-7522-4380-b23e-bc067717c93a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:10.402436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce9427-89bd-466e-90b9-4f1ce38d585d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:10.403334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce9427-89bd-466e-90b9-4f1ce38d585d admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:10.405759 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856f81b-2ecb-4048-b45f-ff1e96a3a908 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:10.406043 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce9427-89bd-466e-90b9-4f1ce38d585d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:10.406043 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce9427-89bd-466e-90b9-4f1ce38d585d admin admin] PATH_INFO: `/v3/projects/93086441bccf456ea67a3cd74a0af05f/users/7854b1efe25046c8a091eeaffdea5b78/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:10.414091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1a51d-b2a9-4c2c-bc97-172145e1bba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:10.414889 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1a51d-b2a9-4c2c-bc97-172145e1bba4 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:10.415237 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1a51d-b2a9-4c2c-bc97-172145e1bba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:10.417154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1a51d-b2a9-4c2c-bc97-172145e1bba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:10.435378 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce9427-89bd-466e-90b9-4f1ce38d585d admin admin] RBAC: Authorizing `identity:create_grant(project_id=93086441bccf456ea67a3cd74a0af05f, user_id=7854b1efe25046c8a091eeaffdea5b78, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:10.435378 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce9427-89bd-466e-90b9-4f1ce38d585d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:10.435714 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1a51d-b2a9-4c2c-bc97-172145e1bba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:10.435714 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1a51d-b2a9-4c2c-bc97-172145e1bba4 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:10.445440 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1268/2535] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:44:10.447134 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:10.517864 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1268/2536] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:10 2022] PUT /identity/v3/projects/93086441bccf456ea67a3cd74a0af05f/users/7854b1efe25046c8a091eeaffdea5b78/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:10.524816 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1a51d-b2a9-4c2c-bc97-172145e1bba4 service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:10.536053 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1a51d-b2a9-4c2c-bc97-172145e1bba4 service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:10.598233 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce9427-89bd-466e-90b9-4f1ce38d585d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:10.614984 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e85334-e03b-4dc9-bc97-f5afd06b6085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:10.616529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e85334-e03b-4dc9-bc97-f5afd06b6085 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:10.617030 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e85334-e03b-4dc9-bc97-f5afd06b6085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:10.617540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e85334-e03b-4dc9-bc97-f5afd06b6085 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:10.618864 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e85334-e03b-4dc9-bc97-f5afd06b6085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:10.622822 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e85334-e03b-4dc9-bc97-f5afd06b6085 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:10.637124 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1269/2537] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:44:10.640975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e85334-e03b-4dc9-bc97-f5afd06b6085 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:10.653122 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e85334-e03b-4dc9-bc97-f5afd06b6085 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:10.658423 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeabe3d-caac-48a3-807b-348a90537de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:10.659291 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeabe3d-caac-48a3-807b-348a90537de9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:10.660467 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeabe3d-caac-48a3-807b-348a90537de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:10.660753 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeabe3d-caac-48a3-807b-348a90537de9 admin admin] PATH_INFO: `/v3/projects/93086441bccf456ea67a3cd74a0af05f/users/7854b1efe25046c8a091eeaffdea5b78/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:10.663056 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeabe3d-caac-48a3-807b-348a90537de9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93086441bccf456ea67a3cd74a0af05f, user_id=7854b1efe25046c8a091eeaffdea5b78, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:10.666748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeabe3d-caac-48a3-807b-348a90537de9 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:10.674339 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce9427-89bd-466e-90b9-4f1ce38d585d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:10.687938 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1270/2538] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:10 2022] PUT /identity/v3/projects/93086441bccf456ea67a3cd74a0af05f/users/7854b1efe25046c8a091eeaffdea5b78/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:10.689541 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104b3c8-74c6-482a-82ce-38f87add6d24 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:10.689789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104b3c8-74c6-482a-82ce-38f87add6d24 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:10.690064 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104b3c8-74c6-482a-82ce-38f87add6d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:10.721883 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-3104b3c8-74c6-482a-82ce-38f87add6d24 None None] MFA Rules not processed for user `7854b1efe25046c8a091eeaffdea5b78`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:10.723928 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-3104b3c8-74c6-482a-82ce-38f87add6d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:10.737416 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc6d38-613e-4a60-a9df-41c3c9d7c922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:10.738119 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc6d38-613e-4a60-a9df-41c3c9d7c922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:10.738595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc6d38-613e-4a60-a9df-41c3c9d7c922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:10.745928 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc6d38-613e-4a60-a9df-41c3c9d7c922 admin admin] PATH_INFO: `/v3/users/c99c5839e57f494f8fcc019dd8e7f4c3` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:10.745928 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc6d38-613e-4a60-a9df-41c3c9d7c922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c99c5839e57f494f8fcc019dd8e7f4c3)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:10.745928 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc6d38-613e-4a60-a9df-41c3c9d7c922 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:10.794968 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1271/2539] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:10.798546 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:10.806208 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104b3c8-74c6-482a-82ce-38f87add6d24 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:10.860428 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104b3c8-74c6-482a-82ce-38f87add6d24 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:10.864287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d46aa-1b8b-48b5-9052-fc0a1c3eaf7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:10.864887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d46aa-1b8b-48b5-9052-fc0a1c3eaf7e service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:10.865206 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d46aa-1b8b-48b5-9052-fc0a1c3eaf7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:10.865482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d46aa-1b8b-48b5-9052-fc0a1c3eaf7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:10.932863 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d46aa-1b8b-48b5-9052-fc0a1c3eaf7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:10.934563 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d46aa-1b8b-48b5-9052-fc0a1c3eaf7e service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:10.944570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1272/2540] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:44:11.067156 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b6dc6d38-613e-4a60-a9df-41c3c9d7c922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c99c5839e57f494f8fcc019dd8e7f4c3'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:11.067895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b6dc6d38-613e-4a60-a9df-41c3c9d7c922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c99c5839e57f494f8fcc019dd8e7f4c3'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:11.255989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b6dc6d38-613e-4a60-a9df-41c3c9d7c922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c99c5839e57f494f8fcc019dd8e7f4c3'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:11.256861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b6dc6d38-613e-4a60-a9df-41c3c9d7c922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c99c5839e57f494f8fcc019dd8e7f4c3'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:11.270895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b6dc6d38-613e-4a60-a9df-41c3c9d7c922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c99c5839e57f494f8fcc019dd8e7f4c3'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:11.283504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1269/2541] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:44:10 2022] DELETE /identity/v3/users/c99c5839e57f494f8fcc019dd8e7f4c3 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:11.288015 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273fa6d-66e2-45c9-aa35-63fee746322d None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:11.288299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273fa6d-66e2-45c9-aa35-63fee746322d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:11.288577 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273fa6d-66e2-45c9-aa35-63fee746322d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:11.308624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-4273fa6d-66e2-45c9-aa35-63fee746322d None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:11.309866 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4273fa6d-66e2-45c9-aa35-63fee746322d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:11.370319 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1273/2542] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:11.380888 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc6d38-613e-4a60-a9df-41c3c9d7c922 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:11.382344 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b6dc6d38-613e-4a60-a9df-41c3c9d7c922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:11.453365 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc6d38-613e-4a60-a9df-41c3c9d7c922 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:11.457049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5d4e6-8ba0-430a-84f9-706e2a75cf6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:11.457902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5d4e6-8ba0-430a-84f9-706e2a75cf6b service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:11.458204 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5d4e6-8ba0-430a-84f9-706e2a75cf6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:11.458477 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5d4e6-8ba0-430a-84f9-706e2a75cf6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:11.468438 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5d4e6-8ba0-430a-84f9-706e2a75cf6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:11.469929 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5d4e6-8ba0-430a-84f9-706e2a75cf6b service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:11.480123 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1270/2543] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:44:11.775213 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:11.961103 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273fa6d-66e2-45c9-aa35-63fee746322d None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:11.962551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4273fa6d-66e2-45c9-aa35-63fee746322d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:11.978388 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273fa6d-66e2-45c9-aa35-63fee746322d None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:11.984173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e1bfa-ea57-4d0b-8a72-486499e9af96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:11.984566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e1bfa-ea57-4d0b-8a72-486499e9af96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:11.984846 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e1bfa-ea57-4d0b-8a72-486499e9af96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:11.985146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e1bfa-ea57-4d0b-8a72-486499e9af96 admin admin] PATH_INFO: `/v3/projects/c507ad8808854f54b328a9a44f62ab7d` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:11.986322 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e1bfa-ea57-4d0b-8a72-486499e9af96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c507ad8808854f54b328a9a44f62ab7d)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:11.988289 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e1bfa-ea57-4d0b-8a72-486499e9af96 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:12.119153 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:12.182293 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-335e1bfa-ea57-4d0b-8a72-486499e9af96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c507ad8808854f54b328a9a44f62ab7d', 'request_id': 'req-335e1bfa-ea57-4d0b-8a72-486499e9af96'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:12.187671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-335e1bfa-ea57-4d0b-8a72-486499e9af96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c507ad8808854f54b328a9a44f62ab7d', 'request_id': 'req-335e1bfa-ea57-4d0b-8a72-486499e9af96'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:12.194266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-335e1bfa-ea57-4d0b-8a72-486499e9af96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c507ad8808854f54b328a9a44f62ab7d', 'request_id': 'req-335e1bfa-ea57-4d0b-8a72-486499e9af96'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:12.318094 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-335e1bfa-ea57-4d0b-8a72-486499e9af96 admin admin] The token cache is being invalidate because project c507ad8808854f54b328a9a44f62ab7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:44:12.318394 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-335e1bfa-ea57-4d0b-8a72-486499e9af96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:12.319003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-335e1bfa-ea57-4d0b-8a72-486499e9af96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:12.320510 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1274/2544] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:44:11 2022] DELETE /identity/v3/projects/c507ad8808854f54b328a9a44f62ab7d => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:12.324704 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5d4e6-8ba0-430a-84f9-706e2a75cf6b service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:12.334403 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5d4e6-8ba0-430a-84f9-706e2a75cf6b service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:12.391921 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b2614-47bb-47cc-a042-c46c1a08c56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:12.392542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b2614-47bb-47cc-a042-c46c1a08c56d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:12.392821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b2614-47bb-47cc-a042-c46c1a08c56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:12.393124 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b2614-47bb-47cc-a042-c46c1a08c56d admin admin] PATH_INFO: `/v3/users/6727a005e30741c08aa0ac7e0e94708c` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:12.394266 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b2614-47bb-47cc-a042-c46c1a08c56d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6727a005e30741c08aa0ac7e0e94708c)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:12.396461 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b2614-47bb-47cc-a042-c46c1a08c56d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:12.536724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:12.652645 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-506b2614-47bb-47cc-a042-c46c1a08c56d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6727a005e30741c08aa0ac7e0e94708c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:12.653331 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-506b2614-47bb-47cc-a042-c46c1a08c56d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6727a005e30741c08aa0ac7e0e94708c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:12.751928 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:12.752263 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-506b2614-47bb-47cc-a042-c46c1a08c56d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6727a005e30741c08aa0ac7e0e94708c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:12.752877 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-506b2614-47bb-47cc-a042-c46c1a08c56d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6727a005e30741c08aa0ac7e0e94708c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:12.767699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-506b2614-47bb-47cc-a042-c46c1a08c56d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6727a005e30741c08aa0ac7e0e94708c'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:12.780631 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1271/2545] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:44:12 2022] DELETE /identity/v3/users/6727a005e30741c08aa0ac7e0e94708c => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:12.848226 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e1bfa-ea57-4d0b-8a72-486499e9af96 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:12.850093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-335e1bfa-ea57-4d0b-8a72-486499e9af96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:12.863114 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e1bfa-ea57-4d0b-8a72-486499e9af96 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:12.925483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b110279-4cbe-44ab-86dd-7d0c656b3960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:12.926271 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b110279-4cbe-44ab-86dd-7d0c656b3960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:12.926648 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b110279-4cbe-44ab-86dd-7d0c656b3960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:12.927032 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b110279-4cbe-44ab-86dd-7d0c656b3960 admin admin] PATH_INFO: `/v3/users/c9a74e34068a47a68cc4e1ed37835c2c` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:12.928168 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b110279-4cbe-44ab-86dd-7d0c656b3960 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9a74e34068a47a68cc4e1ed37835c2c)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:12.930264 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b110279-4cbe-44ab-86dd-7d0c656b3960 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:13.141399 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:13.155928 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2b110279-4cbe-44ab-86dd-7d0c656b3960 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9a74e34068a47a68cc4e1ed37835c2c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:13.296178 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2b110279-4cbe-44ab-86dd-7d0c656b3960 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9a74e34068a47a68cc4e1ed37835c2c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:13.296954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2b110279-4cbe-44ab-86dd-7d0c656b3960 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9a74e34068a47a68cc4e1ed37835c2c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:13.297616 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2b110279-4cbe-44ab-86dd-7d0c656b3960 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9a74e34068a47a68cc4e1ed37835c2c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:13.303858 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b2614-47bb-47cc-a042-c46c1a08c56d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:13.305613 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-506b2614-47bb-47cc-a042-c46c1a08c56d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:13.309819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2b110279-4cbe-44ab-86dd-7d0c656b3960 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9a74e34068a47a68cc4e1ed37835c2c'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:13.320999 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b2614-47bb-47cc-a042-c46c1a08c56d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:13.325910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb643fd4-4bd4-4aa3-8628-b38a0ff38f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:13.326835 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb643fd4-4bd4-4aa3-8628-b38a0ff38f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:13.327321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb643fd4-4bd4-4aa3-8628-b38a0ff38f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:13.327602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb643fd4-4bd4-4aa3-8628-b38a0ff38f09 admin admin] PATH_INFO: `/v3/projects/277e958eb60840ac800db215a2a107f0` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:13.328754 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb643fd4-4bd4-4aa3-8628-b38a0ff38f09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=277e958eb60840ac800db215a2a107f0)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:13.331239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb643fd4-4bd4-4aa3-8628-b38a0ff38f09 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:13.340794 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1275/2546] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:44:12 2022] DELETE /identity/v3/users/c9a74e34068a47a68cc4e1ed37835c2c => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:13.345097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed5a79-f0e9-43a9-8b8b-31a6d986d39b None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:13.345390 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed5a79-f0e9-43a9-8b8b-31a6d986d39b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:13.345669 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed5a79-f0e9-43a9-8b8b-31a6d986d39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:13.364647 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-aeed5a79-f0e9-43a9-8b8b-31a6d986d39b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:13.365901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-aeed5a79-f0e9-43a9-8b8b-31a6d986d39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:13.424569 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1276/2547] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:13.434271 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed5a79-f0e9-43a9-8b8b-31a6d986d39b None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:13.435613 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-aeed5a79-f0e9-43a9-8b8b-31a6d986d39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:13.461873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:13.492588 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed5a79-f0e9-43a9-8b8b-31a6d986d39b None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:13.497018 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf3a33-ee3c-473e-adaa-53973a16219d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:13.497654 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf3a33-ee3c-473e-adaa-53973a16219d service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:13.498067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf3a33-ee3c-473e-adaa-53973a16219d service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:13.498340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf3a33-ee3c-473e-adaa-53973a16219d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:13.509522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf3a33-ee3c-473e-adaa-53973a16219d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:13.511623 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf3a33-ee3c-473e-adaa-53973a16219d service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:13.515736 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bb643fd4-4bd4-4aa3-8628-b38a0ff38f09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '277e958eb60840ac800db215a2a107f0', 'request_id': 'req-bb643fd4-4bd4-4aa3-8628-b38a0ff38f09'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:13.521767 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1277/2548] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:44:13.523803 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bb643fd4-4bd4-4aa3-8628-b38a0ff38f09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '277e958eb60840ac800db215a2a107f0', 'request_id': 'req-bb643fd4-4bd4-4aa3-8628-b38a0ff38f09'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:13.530932 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bb643fd4-4bd4-4aa3-8628-b38a0ff38f09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '277e958eb60840ac800db215a2a107f0', 'request_id': 'req-bb643fd4-4bd4-4aa3-8628-b38a0ff38f09'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:13.655526 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bb643fd4-4bd4-4aa3-8628-b38a0ff38f09 admin admin] The token cache is being invalidate because project 277e958eb60840ac800db215a2a107f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:44:13.655820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bb643fd4-4bd4-4aa3-8628-b38a0ff38f09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:13.656459 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bb643fd4-4bd4-4aa3-8628-b38a0ff38f09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:13.658516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1272/2549] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:44:13 2022] DELETE /identity/v3/projects/277e958eb60840ac800db215a2a107f0 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:13.663096 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41fa7b-e538-4d0e-900e-ae3e2fb9558e None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:13.663383 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41fa7b-e538-4d0e-900e-ae3e2fb9558e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:13.663659 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41fa7b-e538-4d0e-900e-ae3e2fb9558e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:13.684299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-7a41fa7b-e538-4d0e-900e-ae3e2fb9558e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:13.685544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-7a41fa7b-e538-4d0e-900e-ae3e2fb9558e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:13.744842 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1278/2550] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:13.754716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157b762-4b7f-4242-b246-7ac438aa7016 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:13.755086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157b762-4b7f-4242-b246-7ac438aa7016 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:13.755439 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157b762-4b7f-4242-b246-7ac438aa7016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:13.776470 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4157b762-4b7f-4242-b246-7ac438aa7016 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:13.778368 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4157b762-4b7f-4242-b246-7ac438aa7016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:13.786700 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1273/2551] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:13.795784 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41fa7b-e538-4d0e-900e-ae3e2fb9558e None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:13.818712 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41fa7b-e538-4d0e-900e-ae3e2fb9558e None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:13.830173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad8479-8f7a-4512-bb05-e782b112b2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:13.830863 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad8479-8f7a-4512-bb05-e782b112b2fb admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:13.833320 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad8479-8f7a-4512-bb05-e782b112b2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:13.833320 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad8479-8f7a-4512-bb05-e782b112b2fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:13.833626 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad8479-8f7a-4512-bb05-e782b112b2fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:44:13.834036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad8479-8f7a-4512-bb05-e782b112b2fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:13.838125 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad8479-8f7a-4512-bb05-e782b112b2fb admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:13.858694 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1279/2552] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:44:13.865142 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157b762-4b7f-4242-b246-7ac438aa7016 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:13.878497 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157b762-4b7f-4242-b246-7ac438aa7016 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:13.883983 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2586dd-d0e6-4a9f-bf40-04b37ebb5c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:13.890678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2586dd-d0e6-4a9f-bf40-04b37ebb5c42 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:13.890678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2586dd-d0e6-4a9f-bf40-04b37ebb5c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:13.890678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2586dd-d0e6-4a9f-bf40-04b37ebb5c42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:13.890678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2586dd-d0e6-4a9f-bf40-04b37ebb5c42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:13.890678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2586dd-d0e6-4a9f-bf40-04b37ebb5c42 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:13.939887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad8479-8f7a-4512-bb05-e782b112b2fb admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:13.941687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-bdad8479-8f7a-4512-bb05-e782b112b2fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:14.000999 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad8479-8f7a-4512-bb05-e782b112b2fb admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:14.004741 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d050ab8-7429-4677-894a-8e0a072ae3d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:14.005392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d050ab8-7429-4677-894a-8e0a072ae3d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:14.005665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d050ab8-7429-4677-894a-8e0a072ae3d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:14.005941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d050ab8-7429-4677-894a-8e0a072ae3d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:14.007749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-6d050ab8-7429-4677-894a-8e0a072ae3d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:14.021647 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d050ab8-7429-4677-894a-8e0a072ae3d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:14.023097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d050ab8-7429-4677-894a-8e0a072ae3d5 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:14.035661 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1280/2553] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:44:14.036491 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d050ab8-7429-4677-894a-8e0a072ae3d5 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:14.037954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-6d050ab8-7429-4677-894a-8e0a072ae3d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:14.053967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d050ab8-7429-4677-894a-8e0a072ae3d5 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:14.061400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c4ae6-4bdb-4fb4-a6f8-1dc7bd8ff03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:14.062094 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c4ae6-4bdb-4fb4-a6f8-1dc7bd8ff03d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:14.062439 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c4ae6-4bdb-4fb4-a6f8-1dc7bd8ff03d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:14.062773 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c4ae6-4bdb-4fb4-a6f8-1dc7bd8ff03d admin admin] PATH_INFO: `/v3/projects/369e7b4c5eb9401dac0580e4bb4142d4` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:14.063490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1274/2554] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:44:13 2022] POST /identity/v3/projects => generated 409 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:44:14.064834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c4ae6-4bdb-4fb4-a6f8-1dc7bd8ff03d admin admin] RBAC: Authorizing `identity:delete_project(project_id=369e7b4c5eb9401dac0580e4bb4142d4)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:14.068601 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c4ae6-4bdb-4fb4-a6f8-1dc7bd8ff03d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:14.069606 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2586dd-d0e6-4a9f-bf40-04b37ebb5c42 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:14.135484 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2586dd-d0e6-4a9f-bf40-04b37ebb5c42 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:14.141200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0b808-f0f3-4aaf-a9bc-bf388e5f8163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:14.141884 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0b808-f0f3-4aaf-a9bc-bf388e5f8163 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:14.142238 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0b808-f0f3-4aaf-a9bc-bf388e5f8163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:14.142578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0b808-f0f3-4aaf-a9bc-bf388e5f8163 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:14.143285 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0b808-f0f3-4aaf-a9bc-bf388e5f8163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:14.145303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0b808-f0f3-4aaf-a9bc-bf388e5f8163 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:14.223690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-cd2c4ae6-4bdb-4fb4-a6f8-1dc7bd8ff03d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '369e7b4c5eb9401dac0580e4bb4142d4', 'request_id': 'req-cd2c4ae6-4bdb-4fb4-a6f8-1dc7bd8ff03d'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:14.229416 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-cd2c4ae6-4bdb-4fb4-a6f8-1dc7bd8ff03d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '369e7b4c5eb9401dac0580e4bb4142d4', 'request_id': 'req-cd2c4ae6-4bdb-4fb4-a6f8-1dc7bd8ff03d'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:14.235955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-cd2c4ae6-4bdb-4fb4-a6f8-1dc7bd8ff03d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '369e7b4c5eb9401dac0580e4bb4142d4', 'request_id': 'req-cd2c4ae6-4bdb-4fb4-a6f8-1dc7bd8ff03d'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:14.246790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1275/2555] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:44:14 2022] POST /identity/v3/users => generated 305 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:44:14.250984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0b808-f0f3-4aaf-a9bc-bf388e5f8163 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:14.260721 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0b808-f0f3-4aaf-a9bc-bf388e5f8163 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:14.318443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eff666-1dd6-4ba1-98fd-b8301d1b3afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:14.319178 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eff666-1dd6-4ba1-98fd-b8301d1b3afe admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:14.319453 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eff666-1dd6-4ba1-98fd-b8301d1b3afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:14.319794 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eff666-1dd6-4ba1-98fd-b8301d1b3afe admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:14.320428 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eff666-1dd6-4ba1-98fd-b8301d1b3afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:14.322055 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eff666-1dd6-4ba1-98fd-b8301d1b3afe admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:14.333193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1276/2556] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:44:14.337265 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eff666-1dd6-4ba1-98fd-b8301d1b3afe admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:14.347829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eff666-1dd6-4ba1-98fd-b8301d1b3afe admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:14.352959 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ca3c4-f7d2-4b36-9ce5-67a29b7bdd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:14.353241 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:14.355851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ca3c4-f7d2-4b36-9ce5-67a29b7bdd8f admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:14.355851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ca3c4-f7d2-4b36-9ce5-67a29b7bdd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:14.355851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ca3c4-f7d2-4b36-9ce5-67a29b7bdd8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:14.355851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ca3c4-f7d2-4b36-9ce5-67a29b7bdd8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:14.357310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ca3c4-f7d2-4b36-9ce5-67a29b7bdd8f admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:14.361735 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-cd2c4ae6-4bdb-4fb4-a6f8-1dc7bd8ff03d admin admin] The token cache is being invalidate because project 369e7b4c5eb9401dac0580e4bb4142d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:44:14.362209 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-cd2c4ae6-4bdb-4fb4-a6f8-1dc7bd8ff03d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:14.362931 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-cd2c4ae6-4bdb-4fb4-a6f8-1dc7bd8ff03d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:14.365095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1281/2557] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:44:14 2022] DELETE /identity/v3/projects/369e7b4c5eb9401dac0580e4bb4142d4 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:14.369047 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9ec61-9050-444f-923e-7a527c40191f None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:14.369659 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9ec61-9050-444f-923e-7a527c40191f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:14.370097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9ec61-9050-444f-923e-7a527c40191f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:14.371335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1277/2558] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:44:14.372312 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ca3c4-f7d2-4b36-9ce5-67a29b7bdd8f admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:14.374045 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0a7ca3c4-f7d2-4b36-9ce5-67a29b7bdd8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:14.390054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ca3c4-f7d2-4b36-9ce5-67a29b7bdd8f admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:14.392148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b08a9-df4d-4ddc-98cb-0ef940609f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:14.392845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b08a9-df4d-4ddc-98cb-0ef940609f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:14.393267 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b08a9-df4d-4ddc-98cb-0ef940609f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:14.393483 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b08a9-df4d-4ddc-98cb-0ef940609f40 admin admin] PATH_INFO: `/v3/projects/99c78bb5ab694fd19b197e98abf52477/users/f2ed6e2505474ec1afdcf9055e2e80c7/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:14.393690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-1fd9ec61-9050-444f-923e-7a527c40191f None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:14.395073 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-1fd9ec61-9050-444f-923e-7a527c40191f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:14.411106 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1282/2559] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:14.411314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b08a9-df4d-4ddc-98cb-0ef940609f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c78bb5ab694fd19b197e98abf52477, user_id=f2ed6e2505474ec1afdcf9055e2e80c7, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:14.412962 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b08a9-df4d-4ddc-98cb-0ef940609f40 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:14.414410 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467056c-995f-4821-ac60-4817c542c012 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:14.414703 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467056c-995f-4821-ac60-4817c542c012 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:14.415565 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467056c-995f-4821-ac60-4817c542c012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:14.435890 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-c467056c-995f-4821-ac60-4817c542c012 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:14.437138 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c467056c-995f-4821-ac60-4817c542c012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:14.445786 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1283/2560] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:14.450105 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467056c-995f-4821-ac60-4817c542c012 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:14.459774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467056c-995f-4821-ac60-4817c542c012 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:14.465153 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e0b9c-abf2-40da-801e-7135baab0998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:14.465766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e0b9c-abf2-40da-801e-7135baab0998 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:14.466045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e0b9c-abf2-40da-801e-7135baab0998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:14.466318 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e0b9c-abf2-40da-801e-7135baab0998 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:14.466961 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e0b9c-abf2-40da-801e-7135baab0998 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:44:14.467260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e0b9c-abf2-40da-801e-7135baab0998 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:14.468851 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e0b9c-abf2-40da-801e-7135baab0998 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:14.479717 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1278/2561] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:14 2022] PUT /identity/v3/projects/99c78bb5ab694fd19b197e98abf52477/users/f2ed6e2505474ec1afdcf9055e2e80c7/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:14.485848 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1284/2562] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:44:14.487212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b08a9-df4d-4ddc-98cb-0ef940609f40 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:14.491068 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a771df-0b60-41bc-831b-e7ce365ce660 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:14.491349 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a771df-0b60-41bc-831b-e7ce365ce660 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:14.491620 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a771df-0b60-41bc-831b-e7ce365ce660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:14.499313 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:14.519965 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-27a771df-0b60-41bc-831b-e7ce365ce660 None None] MFA Rules not processed for user `f2ed6e2505474ec1afdcf9055e2e80c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:14.521958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-27a771df-0b60-41bc-831b-e7ce365ce660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:14.566464 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b08a9-df4d-4ddc-98cb-0ef940609f40 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:14.571281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da702a-d72e-4c67-98d8-d08f5e40a039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:14.571894 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da702a-d72e-4c67-98d8-d08f5e40a039 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:14.572175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da702a-d72e-4c67-98d8-d08f5e40a039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:14.572454 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da702a-d72e-4c67-98d8-d08f5e40a039 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:14.573138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da702a-d72e-4c67-98d8-d08f5e40a039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:14.575624 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da702a-d72e-4c67-98d8-d08f5e40a039 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:14.604628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1285/2563] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:14 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:14.609846 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a771df-0b60-41bc-831b-e7ce365ce660 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:14.621495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a771df-0b60-41bc-831b-e7ce365ce660 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:14.626420 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83ebf9-4481-43b9-b7cd-b7141c8cc14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:14.627036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83ebf9-4481-43b9-b7cd-b7141c8cc14d admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:14.627460 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83ebf9-4481-43b9-b7cd-b7141c8cc14d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:14.627734 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83ebf9-4481-43b9-b7cd-b7141c8cc14d admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:14.628383 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83ebf9-4481-43b9-b7cd-b7141c8cc14d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:14.630350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83ebf9-4481-43b9-b7cd-b7141c8cc14d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:14.668714 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1279/2564] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:44:14 2022] POST /identity/v3/projects => generated 375 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:44:14.673354 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da702a-d72e-4c67-98d8-d08f5e40a039 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:14.760566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da702a-d72e-4c67-98d8-d08f5e40a039 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:14.766303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca93247-1e4f-4f9c-a049-17c0916805f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:14.767115 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca93247-1e4f-4f9c-a049-17c0916805f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:14.767473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca93247-1e4f-4f9c-a049-17c0916805f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:14.767718 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca93247-1e4f-4f9c-a049-17c0916805f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:14.768369 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca93247-1e4f-4f9c-a049-17c0916805f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:14.770679 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca93247-1e4f-4f9c-a049-17c0916805f6 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:15.032187 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1286/2565] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:44:14 2022] POST /identity/v3/projects => generated 409 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:44:15.033295 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83ebf9-4481-43b9-b7cd-b7141c8cc14d admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:15.091734 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1280/2566] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:44:14 2022] POST /identity/v3/users => generated 288 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:44:15.095834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca93247-1e4f-4f9c-a049-17c0916805f6 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:15.100733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83ebf9-4481-43b9-b7cd-b7141c8cc14d admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:15.105644 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca93247-1e4f-4f9c-a049-17c0916805f6 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:15.105798 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32125244-7eea-4f7b-8002-14f8c554a9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:15.106278 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32125244-7eea-4f7b-8002-14f8c554a9f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:15.106560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32125244-7eea-4f7b-8002-14f8c554a9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:15.106831 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32125244-7eea-4f7b-8002-14f8c554a9f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:15.107574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32125244-7eea-4f7b-8002-14f8c554a9f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:15.109537 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32125244-7eea-4f7b-8002-14f8c554a9f9 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:15.111128 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f76f5-947e-42af-970c-884e0df230a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:15.111799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f76f5-947e-42af-970c-884e0df230a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:15.112143 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f76f5-947e-42af-970c-884e0df230a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:15.112479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f76f5-947e-42af-970c-884e0df230a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:15.113216 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f76f5-947e-42af-970c-884e0df230a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:15.114891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f76f5-947e-42af-970c-884e0df230a9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:15.126232 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1281/2567] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:44:15.130250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f76f5-947e-42af-970c-884e0df230a9 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:15.140423 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f76f5-947e-42af-970c-884e0df230a9 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:15.144989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3661d54-76cd-427b-9445-8a764dc45bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:15.145651 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3661d54-76cd-427b-9445-8a764dc45bda admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:15.145916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3661d54-76cd-427b-9445-8a764dc45bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:15.146440 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3661d54-76cd-427b-9445-8a764dc45bda admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:15.147061 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3661d54-76cd-427b-9445-8a764dc45bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:15.148619 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3661d54-76cd-427b-9445-8a764dc45bda admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:15.158240 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1282/2568] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:44:15.162118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3661d54-76cd-427b-9445-8a764dc45bda admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:15.173146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3661d54-76cd-427b-9445-8a764dc45bda admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:15.175739 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3899e5-f436-4096-8c50-11c22f91decf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:15.176338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3899e5-f436-4096-8c50-11c22f91decf admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:15.176618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3899e5-f436-4096-8c50-11c22f91decf admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:15.176890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3899e5-f436-4096-8c50-11c22f91decf admin admin] PATH_INFO: `/v3/projects/3e018cfe2e04479d92ba517315b92c8b/users/d04598e263fa445f81d33287aba03cb4/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:15.191423 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3899e5-f436-4096-8c50-11c22f91decf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e018cfe2e04479d92ba517315b92c8b, user_id=d04598e263fa445f81d33287aba03cb4, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:15.194082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3899e5-f436-4096-8c50-11c22f91decf admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:15.248204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1287/2569] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:44:15 2022] POST /identity/v3/users => generated 311 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:44:15.249106 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32125244-7eea-4f7b-8002-14f8c554a9f9 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:15.259026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32125244-7eea-4f7b-8002-14f8c554a9f9 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:15.264446 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a8d09-de1e-4fe0-ae0c-523e99ec04f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:15.265049 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a8d09-de1e-4fe0-ae0c-523e99ec04f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:15.265554 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a8d09-de1e-4fe0-ae0c-523e99ec04f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:15.265834 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a8d09-de1e-4fe0-ae0c-523e99ec04f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:15.266483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a8d09-de1e-4fe0-ae0c-523e99ec04f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:15.268086 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a8d09-de1e-4fe0-ae0c-523e99ec04f6 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:15.283388 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1288/2570] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:44:15.285465 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a8d09-de1e-4fe0-ae0c-523e99ec04f6 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:15.297049 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a8d09-de1e-4fe0-ae0c-523e99ec04f6 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:15.302027 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b176204-b550-44a4-b4a3-f2a5389c2120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:15.302844 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b176204-b550-44a4-b4a3-f2a5389c2120 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:15.303250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b176204-b550-44a4-b4a3-f2a5389c2120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:15.303728 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b176204-b550-44a4-b4a3-f2a5389c2120 admin admin] PATH_INFO: `/v3/projects/16c8f4f591ed43819b20b8a92f077d77/users/0f7a6d0a6d96465caaccf9b457afbbc5/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:15.323782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b176204-b550-44a4-b4a3-f2a5389c2120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16c8f4f591ed43819b20b8a92f077d77, user_id=0f7a6d0a6d96465caaccf9b457afbbc5, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:15.328518 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b176204-b550-44a4-b4a3-f2a5389c2120 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:15.360233 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1283/2571] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:15 2022] PUT /identity/v3/projects/3e018cfe2e04479d92ba517315b92c8b/users/d04598e263fa445f81d33287aba03cb4/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:15.372575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42491952-4798-4369-a582-6ce0db4831e9 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:15.372896 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42491952-4798-4369-a582-6ce0db4831e9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:15.373434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42491952-4798-4369-a582-6ce0db4831e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:15.407187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-42491952-4798-4369-a582-6ce0db4831e9 None None] MFA Rules not processed for user `d04598e263fa445f81d33287aba03cb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:15.408738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-42491952-4798-4369-a582-6ce0db4831e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:15.464488 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1289/2572] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:15 2022] PUT /identity/v3/projects/16c8f4f591ed43819b20b8a92f077d77/users/0f7a6d0a6d96465caaccf9b457afbbc5/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:15.466205 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b176204-b550-44a4-b4a3-f2a5389c2120 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:15.479372 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:15.498536 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1284/2573] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:15.509161 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42491952-4798-4369-a582-6ce0db4831e9 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:15.510649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-42491952-4798-4369-a582-6ce0db4831e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:15.538132 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b176204-b550-44a4-b4a3-f2a5389c2120 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:15.543674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b62064-9b24-4099-b942-d05038ef78eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:15.546983 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b62064-9b24-4099-b942-d05038ef78eb admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:15.546983 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b62064-9b24-4099-b942-d05038ef78eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:15.546983 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b62064-9b24-4099-b942-d05038ef78eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:15.546983 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b62064-9b24-4099-b942-d05038ef78eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:15.547401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b62064-9b24-4099-b942-d05038ef78eb admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:15.557704 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1290/2574] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:44:15.562695 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b62064-9b24-4099-b942-d05038ef78eb admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:15.571522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:15.575803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b62064-9b24-4099-b942-d05038ef78eb admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:15.577862 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985ec0e-0dde-4687-8104-f9079c5c3744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:15.578672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985ec0e-0dde-4687-8104-f9079c5c3744 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:15.579052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985ec0e-0dde-4687-8104-f9079c5c3744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:15.579418 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42491952-4798-4369-a582-6ce0db4831e9 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:15.579573 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985ec0e-0dde-4687-8104-f9079c5c3744 admin admin] PATH_INFO: `/v3/projects/16c8f4f591ed43819b20b8a92f077d77/users/0f7a6d0a6d96465caaccf9b457afbbc5/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:15.581737 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985ec0e-0dde-4687-8104-f9079c5c3744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16c8f4f591ed43819b20b8a92f077d77, user_id=0f7a6d0a6d96465caaccf9b457afbbc5, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:15.584592 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985ec0e-0dde-4687-8104-f9079c5c3744 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:15.596340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1291/2575] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:15 2022] PUT /identity/v3/projects/16c8f4f591ed43819b20b8a92f077d77/users/0f7a6d0a6d96465caaccf9b457afbbc5/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:15.607216 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34473e4e-7e45-4f8d-8952-1a843f11a91f None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:15.607917 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34473e4e-7e45-4f8d-8952-1a843f11a91f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:15.607917 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34473e4e-7e45-4f8d-8952-1a843f11a91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:15.635311 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeddec1-1ab6-485a-87f1-da1343029c97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:15.636067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeddec1-1ab6-485a-87f1-da1343029c97 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:15.637106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeddec1-1ab6-485a-87f1-da1343029c97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:15.637386 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeddec1-1ab6-485a-87f1-da1343029c97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:15.637754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-34473e4e-7e45-4f8d-8952-1a843f11a91f None None] MFA Rules not processed for user `0f7a6d0a6d96465caaccf9b457afbbc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:15.639160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-34473e4e-7e45-4f8d-8952-1a843f11a91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:15.698979 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeddec1-1ab6-485a-87f1-da1343029c97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:15.700522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeddec1-1ab6-485a-87f1-da1343029c97 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:15.712565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1285/2576] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:44:15.715170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1292/2577] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:15 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:15.736142 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeddec1-1ab6-485a-87f1-da1343029c97 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:15.736142 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-faeddec1-1ab6-485a-87f1-da1343029c97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:15.739943 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeddec1-1ab6-485a-87f1-da1343029c97 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:15.807073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdff11a-126b-487e-8718-d030dbe0adc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:15.807073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdff11a-126b-487e-8718-d030dbe0adc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:15.807073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdff11a-126b-487e-8718-d030dbe0adc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:15.807073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdff11a-126b-487e-8718-d030dbe0adc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:15.816294 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdff11a-126b-487e-8718-d030dbe0adc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:15.817933 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdff11a-126b-487e-8718-d030dbe0adc2 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:15.828538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1286/2578] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:15 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:44:15.959248 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:16.278283 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:16.520682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34473e4e-7e45-4f8d-8952-1a843f11a91f None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:16.533556 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34473e4e-7e45-4f8d-8952-1a843f11a91f None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:16.537270 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75e284-4fd7-4cf1-bae3-855426be4c5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:16.537873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75e284-4fd7-4cf1-bae3-855426be4c5f service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:16.538152 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75e284-4fd7-4cf1-bae3-855426be4c5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:16.538425 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75e284-4fd7-4cf1-bae3-855426be4c5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:16.595733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75e284-4fd7-4cf1-bae3-855426be4c5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:16.597208 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75e284-4fd7-4cf1-bae3-855426be4c5f service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:16.606070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1293/2579] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:16 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:44:17.314778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:17.562156 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdff11a-126b-487e-8718-d030dbe0adc2 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:17.571638 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdff11a-126b-487e-8718-d030dbe0adc2 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:17.575446 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec945f-ccad-4d94-bb8e-e7d33fd1d19c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:17.576041 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec945f-ccad-4d94-bb8e-e7d33fd1d19c service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:17.576318 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec945f-ccad-4d94-bb8e-e7d33fd1d19c service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:17.576591 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec945f-ccad-4d94-bb8e-e7d33fd1d19c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:17.578386 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-50ec945f-ccad-4d94-bb8e-e7d33fd1d19c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:17.659196 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec945f-ccad-4d94-bb8e-e7d33fd1d19c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:17.661434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec945f-ccad-4d94-bb8e-e7d33fd1d19c service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:17.689416 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1287/2580] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:44:17.805230 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:18.208970 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75e284-4fd7-4cf1-bae3-855426be4c5f service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:18.211248 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-3c75e284-4fd7-4cf1-bae3-855426be4c5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:18.286154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75e284-4fd7-4cf1-bae3-855426be4c5f service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:18.290740 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151526f0-90b8-4d49-83c2-99f43c63dce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:18.291372 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151526f0-90b8-4d49-83c2-99f43c63dce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:18.291714 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151526f0-90b8-4d49-83c2-99f43c63dce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:18.291994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151526f0-90b8-4d49-83c2-99f43c63dce1 admin admin] PATH_INFO: `/v3/users/d4d9bea213fa4401992d8e7894726e05` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:18.293174 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151526f0-90b8-4d49-83c2-99f43c63dce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4d9bea213fa4401992d8e7894726e05)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:18.295164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151526f0-90b8-4d49-83c2-99f43c63dce1 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:18.468940 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-151526f0-90b8-4d49-83c2-99f43c63dce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4d9bea213fa4401992d8e7894726e05'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:18.552511 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-151526f0-90b8-4d49-83c2-99f43c63dce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4d9bea213fa4401992d8e7894726e05'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:18.553185 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-151526f0-90b8-4d49-83c2-99f43c63dce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4d9bea213fa4401992d8e7894726e05'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:18.553819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-151526f0-90b8-4d49-83c2-99f43c63dce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4d9bea213fa4401992d8e7894726e05'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:18.562167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-151526f0-90b8-4d49-83c2-99f43c63dce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4d9bea213fa4401992d8e7894726e05'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:18.592133 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1294/2581] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:44:18 2022] DELETE /identity/v3/users/d4d9bea213fa4401992d8e7894726e05 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:18.874653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec945f-ccad-4d94-bb8e-e7d33fd1d19c service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:18.886154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec945f-ccad-4d94-bb8e-e7d33fd1d19c service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:18.946224 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1acbd-605e-4f27-9e1f-c0d692e02787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:18.946856 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1acbd-605e-4f27-9e1f-c0d692e02787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:18.947136 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1acbd-605e-4f27-9e1f-c0d692e02787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:18.947412 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1acbd-605e-4f27-9e1f-c0d692e02787 admin admin] PATH_INFO: `/v3/projects/e3160b641b81433780d38eca696bfdf1` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:18.948541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1acbd-605e-4f27-9e1f-c0d692e02787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3160b641b81433780d38eca696bfdf1)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:18.950504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1acbd-605e-4f27-9e1f-c0d692e02787 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:18.993301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:19.079123 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-34b1acbd-605e-4f27-9e1f-c0d692e02787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3160b641b81433780d38eca696bfdf1', 'request_id': 'req-34b1acbd-605e-4f27-9e1f-c0d692e02787'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:19.084949 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-34b1acbd-605e-4f27-9e1f-c0d692e02787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3160b641b81433780d38eca696bfdf1', 'request_id': 'req-34b1acbd-605e-4f27-9e1f-c0d692e02787'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:19.091577 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-34b1acbd-605e-4f27-9e1f-c0d692e02787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3160b641b81433780d38eca696bfdf1', 'request_id': 'req-34b1acbd-605e-4f27-9e1f-c0d692e02787'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:19.164515 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-34b1acbd-605e-4f27-9e1f-c0d692e02787 admin admin] The token cache is being invalidate because project e3160b641b81433780d38eca696bfdf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:44:19.164834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-34b1acbd-605e-4f27-9e1f-c0d692e02787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:19.165484 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-34b1acbd-605e-4f27-9e1f-c0d692e02787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:19.167009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1288/2582] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:44:18 2022] DELETE /identity/v3/projects/e3160b641b81433780d38eca696bfdf1 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:19.171934 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bdc76-7513-41a9-b624-5d0cacfeaf0e None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:19.172223 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bdc76-7513-41a9-b624-5d0cacfeaf0e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:19.172503 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bdc76-7513-41a9-b624-5d0cacfeaf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:19.192734 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-174bdc76-7513-41a9-b624-5d0cacfeaf0e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:19.194009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-174bdc76-7513-41a9-b624-5d0cacfeaf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:19.252947 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1295/2583] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:19.260990 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aef1f2-c6b6-4da1-8ca3-bd5bf2548c16 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:19.261312 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aef1f2-c6b6-4da1-8ca3-bd5bf2548c16 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:19.261595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aef1f2-c6b6-4da1-8ca3-bd5bf2548c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:19.269474 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:19.284041 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a0aef1f2-c6b6-4da1-8ca3-bd5bf2548c16 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:19.285282 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a0aef1f2-c6b6-4da1-8ca3-bd5bf2548c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:19.293270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1289/2584] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:19.297508 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bdc76-7513-41a9-b624-5d0cacfeaf0e None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:19.307801 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bdc76-7513-41a9-b624-5d0cacfeaf0e None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:19.312478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd426af1-1b31-4435-a096-409a8e298ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:19.313125 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd426af1-1b31-4435-a096-409a8e298ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:19.313394 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd426af1-1b31-4435-a096-409a8e298ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:19.313693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd426af1-1b31-4435-a096-409a8e298ef8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:19.314346 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd426af1-1b31-4435-a096-409a8e298ef8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:44:19.314650 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd426af1-1b31-4435-a096-409a8e298ef8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:19.316381 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd426af1-1b31-4435-a096-409a8e298ef8 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:19.332541 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1296/2585] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:44:19.333714 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aef1f2-c6b6-4da1-8ca3-bd5bf2548c16 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:19.343264 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aef1f2-c6b6-4da1-8ca3-bd5bf2548c16 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:19.348905 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d2e41-8f3b-4e55-9c65-e2ceea57c799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:19.352109 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d2e41-8f3b-4e55-9c65-e2ceea57c799 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:19.352109 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d2e41-8f3b-4e55-9c65-e2ceea57c799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:19.352109 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d2e41-8f3b-4e55-9c65-e2ceea57c799 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:19.352109 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d2e41-8f3b-4e55-9c65-e2ceea57c799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:19.352669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d2e41-8f3b-4e55-9c65-e2ceea57c799 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:19.458321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1290/2586] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:44:19 2022] POST /identity/v3/projects => generated 367 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:44:19.462752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd426af1-1b31-4435-a096-409a8e298ef8 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:19.473340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd426af1-1b31-4435-a096-409a8e298ef8 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:19.495817 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:19.532002 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be180ff-aec2-4c66-9b3d-8ac2e6bda990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:19.532713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be180ff-aec2-4c66-9b3d-8ac2e6bda990 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:19.533622 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be180ff-aec2-4c66-9b3d-8ac2e6bda990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:19.533906 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be180ff-aec2-4c66-9b3d-8ac2e6bda990 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:19.534581 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be180ff-aec2-4c66-9b3d-8ac2e6bda990 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:19.536507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be180ff-aec2-4c66-9b3d-8ac2e6bda990 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:19.641196 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1297/2587] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:44:19 2022] POST /identity/v3/users => generated 284 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:44:19.647889 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d2e41-8f3b-4e55-9c65-e2ceea57c799 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:19.658948 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d2e41-8f3b-4e55-9c65-e2ceea57c799 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:19.663603 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4708ad7-5133-4efa-85f2-2ecc32bca342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:19.664200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4708ad7-5133-4efa-85f2-2ecc32bca342 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:19.664480 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4708ad7-5133-4efa-85f2-2ecc32bca342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:19.664781 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4708ad7-5133-4efa-85f2-2ecc32bca342 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:19.665481 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4708ad7-5133-4efa-85f2-2ecc32bca342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:19.667059 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4708ad7-5133-4efa-85f2-2ecc32bca342 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:19.677804 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1291/2588] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:44:19.681798 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be180ff-aec2-4c66-9b3d-8ac2e6bda990 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:19.692497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be180ff-aec2-4c66-9b3d-8ac2e6bda990 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:19.701698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bdc48-4c8a-4449-bc58-b8c806cd5db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:19.702307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bdc48-4c8a-4449-bc58-b8c806cd5db9 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:19.702588 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bdc48-4c8a-4449-bc58-b8c806cd5db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:19.702868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bdc48-4c8a-4449-bc58-b8c806cd5db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:19.703509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3bdc48-4c8a-4449-bc58-b8c806cd5db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:19.705214 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3bdc48-4c8a-4449-bc58-b8c806cd5db9 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:19.716254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1298/2589] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:44:19.721006 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4708ad7-5133-4efa-85f2-2ecc32bca342 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:19.730916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4708ad7-5133-4efa-85f2-2ecc32bca342 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:19.736865 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9105841-a626-40dc-af1f-a8b965c54c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:19.737592 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9105841-a626-40dc-af1f-a8b965c54c5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:19.744087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9105841-a626-40dc-af1f-a8b965c54c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:19.744087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9105841-a626-40dc-af1f-a8b965c54c5d admin admin] PATH_INFO: `/v3/projects/8eed977aa4ec4ce4b7898d35b37b5896/users/fc3065685ea847f9a823efd2d12674cf/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:19.752311 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9105841-a626-40dc-af1f-a8b965c54c5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eed977aa4ec4ce4b7898d35b37b5896, user_id=fc3065685ea847f9a823efd2d12674cf, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:19.754957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9105841-a626-40dc-af1f-a8b965c54c5d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:19.850653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1292/2590] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:19 2022] PUT /identity/v3/projects/8eed977aa4ec4ce4b7898d35b37b5896/users/fc3065685ea847f9a823efd2d12674cf/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:19.862580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162e4fd-096d-4e92-aca5-999afb95759b None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:19.862872 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162e4fd-096d-4e92-aca5-999afb95759b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:19.863178 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162e4fd-096d-4e92-aca5-999afb95759b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:19.890624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-3162e4fd-096d-4e92-aca5-999afb95759b None None] MFA Rules not processed for user `fc3065685ea847f9a823efd2d12674cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:19.891811 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-3162e4fd-096d-4e92-aca5-999afb95759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:19.955500 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1299/2591] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:19.966547 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9105841-a626-40dc-af1f-a8b965c54c5d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:19.968217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d9105841-a626-40dc-af1f-a8b965c54c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:20.028399 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9105841-a626-40dc-af1f-a8b965c54c5d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:20.032191 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba39f2d-71f7-42c2-a55b-dd5350a96f03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:20.033386 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba39f2d-71f7-42c2-a55b-dd5350a96f03 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:20.033666 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba39f2d-71f7-42c2-a55b-dd5350a96f03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:20.033936 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba39f2d-71f7-42c2-a55b-dd5350a96f03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:20.042165 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba39f2d-71f7-42c2-a55b-dd5350a96f03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:20.043600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba39f2d-71f7-42c2-a55b-dd5350a96f03 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:20.053081 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1293/2592] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:44:30.048415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=52,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:30.153634 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=60,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:33.614530 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162e4fd-096d-4e92-aca5-999afb95759b None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:33.616080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-3162e4fd-096d-4e92-aca5-999afb95759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:33.679614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162e4fd-096d-4e92-aca5-999afb95759b None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:33.684129 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5d73f-b22c-4897-a9f6-724cd37f9328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:33.684754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5d73f-b22c-4897-a9f6-724cd37f9328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:33.685037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5d73f-b22c-4897-a9f6-724cd37f9328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:33.685348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5d73f-b22c-4897-a9f6-724cd37f9328 admin admin] PATH_INFO: `/v3/users/f2ed6e2505474ec1afdcf9055e2e80c7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:33.686480 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd5d73f-b22c-4897-a9f6-724cd37f9328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2ed6e2505474ec1afdcf9055e2e80c7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:33.688441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd5d73f-b22c-4897-a9f6-724cd37f9328 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:33.879924 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-bbd5d73f-b22c-4897-a9f6-724cd37f9328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2ed6e2505474ec1afdcf9055e2e80c7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:33.951477 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-bbd5d73f-b22c-4897-a9f6-724cd37f9328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2ed6e2505474ec1afdcf9055e2e80c7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:33.952098 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-bbd5d73f-b22c-4897-a9f6-724cd37f9328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2ed6e2505474ec1afdcf9055e2e80c7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:33.952690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-bbd5d73f-b22c-4897-a9f6-724cd37f9328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2ed6e2505474ec1afdcf9055e2e80c7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:33.960925 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-bbd5d73f-b22c-4897-a9f6-724cd37f9328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2ed6e2505474ec1afdcf9055e2e80c7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:33.978331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1300/2593] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:44:33 2022] DELETE /identity/v3/users/f2ed6e2505474ec1afdcf9055e2e80c7 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:33.982930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d85919-7f82-49bd-b1f1-1fae0610fc91 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:33.983449 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d85919-7f82-49bd-b1f1-1fae0610fc91 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:33.983548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d85919-7f82-49bd-b1f1-1fae0610fc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:34.002248 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f1d85919-7f82-49bd-b1f1-1fae0610fc91 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:34.003988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f1d85919-7f82-49bd-b1f1-1fae0610fc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:34.061563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1294/2594] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:34.071957 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5d73f-b22c-4897-a9f6-724cd37f9328 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:34.073395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-bbd5d73f-b22c-4897-a9f6-724cd37f9328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:34.129474 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5d73f-b22c-4897-a9f6-724cd37f9328 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:34.133036 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d325e52-b9b3-4260-a9de-3dc09c87735a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:34.133663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d325e52-b9b3-4260-a9de-3dc09c87735a service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:34.133940 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d325e52-b9b3-4260-a9de-3dc09c87735a service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:34.134215 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d325e52-b9b3-4260-a9de-3dc09c87735a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:34.143038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d325e52-b9b3-4260-a9de-3dc09c87735a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:34.145666 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d325e52-b9b3-4260-a9de-3dc09c87735a service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:34.154896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1301/2595] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:44:34.572175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d85919-7f82-49bd-b1f1-1fae0610fc91 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:34.573662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f1d85919-7f82-49bd-b1f1-1fae0610fc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:34.588183 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d85919-7f82-49bd-b1f1-1fae0610fc91 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:34.593239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafb712-3666-48e1-a253-98abf9a4db68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:34.593876 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafb712-3666-48e1-a253-98abf9a4db68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:34.594157 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafb712-3666-48e1-a253-98abf9a4db68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:34.594640 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafb712-3666-48e1-a253-98abf9a4db68 admin admin] PATH_INFO: `/v3/projects/99c78bb5ab694fd19b197e98abf52477` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:34.595878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafb712-3666-48e1-a253-98abf9a4db68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99c78bb5ab694fd19b197e98abf52477)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:34.598071 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafb712-3666-48e1-a253-98abf9a4db68 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:34.707544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bcafb712-3666-48e1-a253-98abf9a4db68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99c78bb5ab694fd19b197e98abf52477', 'request_id': 'req-bcafb712-3666-48e1-a253-98abf9a4db68'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:34.712951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bcafb712-3666-48e1-a253-98abf9a4db68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99c78bb5ab694fd19b197e98abf52477', 'request_id': 'req-bcafb712-3666-48e1-a253-98abf9a4db68'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:34.719007 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bcafb712-3666-48e1-a253-98abf9a4db68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99c78bb5ab694fd19b197e98abf52477', 'request_id': 'req-bcafb712-3666-48e1-a253-98abf9a4db68'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:34.786972 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bcafb712-3666-48e1-a253-98abf9a4db68 admin admin] The token cache is being invalidate because project 99c78bb5ab694fd19b197e98abf52477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:44:34.787263 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bcafb712-3666-48e1-a253-98abf9a4db68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:34.787874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bcafb712-3666-48e1-a253-98abf9a4db68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:34.789360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1295/2596] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:44:34 2022] DELETE /identity/v3/projects/99c78bb5ab694fd19b197e98abf52477 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:34.793366 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d325e52-b9b3-4260-a9de-3dc09c87735a service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:34.803523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d325e52-b9b3-4260-a9de-3dc09c87735a service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:34.822337 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:34.863108 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a83a3c-378c-46ac-81db-c46376dd528f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:34.863738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a83a3c-378c-46ac-81db-c46376dd528f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:34.864017 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a83a3c-378c-46ac-81db-c46376dd528f admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:34.864292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a83a3c-378c-46ac-81db-c46376dd528f admin admin] PATH_INFO: `/v3/users/0f7a6d0a6d96465caaccf9b457afbbc5` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:34.865441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a83a3c-378c-46ac-81db-c46376dd528f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f7a6d0a6d96465caaccf9b457afbbc5)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:34.867409 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a83a3c-378c-46ac-81db-c46376dd528f admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:35.092692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f7a83a3c-378c-46ac-81db-c46376dd528f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f7a6d0a6d96465caaccf9b457afbbc5'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:35.181526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f7a83a3c-378c-46ac-81db-c46376dd528f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f7a6d0a6d96465caaccf9b457afbbc5'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:35.182156 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f7a83a3c-378c-46ac-81db-c46376dd528f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f7a6d0a6d96465caaccf9b457afbbc5'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:35.182795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f7a83a3c-378c-46ac-81db-c46376dd528f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f7a6d0a6d96465caaccf9b457afbbc5'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:35.190381 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-f7a83a3c-378c-46ac-81db-c46376dd528f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f7a6d0a6d96465caaccf9b457afbbc5'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:35.210171 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1302/2597] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:44:34 2022] DELETE /identity/v3/users/0f7a6d0a6d96465caaccf9b457afbbc5 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:35.594947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafb712-3666-48e1-a253-98abf9a4db68 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:35.596574 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bcafb712-3666-48e1-a253-98abf9a4db68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:35.612122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafb712-3666-48e1-a253-98abf9a4db68 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:35.671211 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c55893-cc71-419a-850f-a916510fc0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:35.671859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c55893-cc71-419a-850f-a916510fc0b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:35.672137 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c55893-cc71-419a-850f-a916510fc0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:35.672406 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c55893-cc71-419a-850f-a916510fc0b6 admin admin] PATH_INFO: `/v3/projects/16c8f4f591ed43819b20b8a92f077d77` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:35.673524 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c55893-cc71-419a-850f-a916510fc0b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16c8f4f591ed43819b20b8a92f077d77)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:35.675463 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c55893-cc71-419a-850f-a916510fc0b6 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:35.777236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:35.793104 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a6c55893-cc71-419a-850f-a916510fc0b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16c8f4f591ed43819b20b8a92f077d77', 'request_id': 'req-a6c55893-cc71-419a-850f-a916510fc0b6'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:35.798454 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a6c55893-cc71-419a-850f-a916510fc0b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16c8f4f591ed43819b20b8a92f077d77', 'request_id': 'req-a6c55893-cc71-419a-850f-a916510fc0b6'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:35.804899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a6c55893-cc71-419a-850f-a916510fc0b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16c8f4f591ed43819b20b8a92f077d77', 'request_id': 'req-a6c55893-cc71-419a-850f-a916510fc0b6'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:35.894275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a6c55893-cc71-419a-850f-a916510fc0b6 admin admin] The token cache is being invalidate because project 16c8f4f591ed43819b20b8a92f077d77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:44:35.894494 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a6c55893-cc71-419a-850f-a916510fc0b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:35.895118 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a6c55893-cc71-419a-850f-a916510fc0b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:35.896907 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1296/2598] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:44:35 2022] DELETE /identity/v3/projects/16c8f4f591ed43819b20b8a92f077d77 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:35.901783 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b504c-a83b-4aef-b5a5-69978be289c2 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:35.902260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b504c-a83b-4aef-b5a5-69978be289c2 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:35.902385 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b504c-a83b-4aef-b5a5-69978be289c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:35.922071 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-203b504c-a83b-4aef-b5a5-69978be289c2 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:35.923745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-203b504c-a83b-4aef-b5a5-69978be289c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:35.987695 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1303/2599] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:35.996523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c07b5-52b4-43ae-8b2c-4565c8f7428a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:35.996811 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c07b5-52b4-43ae-8b2c-4565c8f7428a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:35.997123 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c07b5-52b4-43ae-8b2c-4565c8f7428a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:36.017454 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-071c07b5-52b4-43ae-8b2c-4565c8f7428a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:36.018678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-071c07b5-52b4-43ae-8b2c-4565c8f7428a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:36.026809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1297/2600] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:36.031127 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b504c-a83b-4aef-b5a5-69978be289c2 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:36.040593 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b504c-a83b-4aef-b5a5-69978be289c2 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:36.045271 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f406e61-bf06-42c7-b0df-0028de82a6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:36.045875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f406e61-bf06-42c7-b0df-0028de82a6cd admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:36.046155 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f406e61-bf06-42c7-b0df-0028de82a6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:36.046435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f406e61-bf06-42c7-b0df-0028de82a6cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:36.047107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f406e61-bf06-42c7-b0df-0028de82a6cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:44:36.047441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f406e61-bf06-42c7-b0df-0028de82a6cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:36.049042 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f406e61-bf06-42c7-b0df-0028de82a6cd admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:36.061012 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1304/2601] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:44:36.066035 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c07b5-52b4-43ae-8b2c-4565c8f7428a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:36.075557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c07b5-52b4-43ae-8b2c-4565c8f7428a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:36.081860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406af5b7-3fe1-423b-8171-409ff80fa5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:36.081860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406af5b7-3fe1-423b-8171-409ff80fa5ea admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:36.081860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406af5b7-3fe1-423b-8171-409ff80fa5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:36.081860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406af5b7-3fe1-423b-8171-409ff80fa5ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:36.082311 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406af5b7-3fe1-423b-8171-409ff80fa5ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:36.084205 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406af5b7-3fe1-423b-8171-409ff80fa5ea admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:36.212327 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1298/2602] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:44:36 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:44:36.216654 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f406e61-bf06-42c7-b0df-0028de82a6cd admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:36.226702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f406e61-bf06-42c7-b0df-0028de82a6cd admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:36.254569 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:36.286395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd60ce-e32d-4f65-843c-e90db5566867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:36.287305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd60ce-e32d-4f65-843c-e90db5566867 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:36.287865 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd60ce-e32d-4f65-843c-e90db5566867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:36.287865 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd60ce-e32d-4f65-843c-e90db5566867 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:36.288497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd60ce-e32d-4f65-843c-e90db5566867 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:36.290546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd60ce-e32d-4f65-843c-e90db5566867 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:36.422342 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1305/2603] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:44:36 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:44:36.429492 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406af5b7-3fe1-423b-8171-409ff80fa5ea admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:36.440690 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406af5b7-3fe1-423b-8171-409ff80fa5ea admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:36.445318 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274f9b3-9279-41de-b524-d72f4c88cb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:36.445920 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274f9b3-9279-41de-b524-d72f4c88cb44 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:36.446200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274f9b3-9279-41de-b524-d72f4c88cb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:36.446501 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274f9b3-9279-41de-b524-d72f4c88cb44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:36.447166 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274f9b3-9279-41de-b524-d72f4c88cb44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:36.448809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274f9b3-9279-41de-b524-d72f4c88cb44 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:36.459291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1299/2604] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:44:36.463639 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd60ce-e32d-4f65-843c-e90db5566867 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:36.474345 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd60ce-e32d-4f65-843c-e90db5566867 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:36.479169 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892654c-9d80-4827-9345-837b60d34358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:36.479772 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892654c-9d80-4827-9345-837b60d34358 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:36.480068 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892654c-9d80-4827-9345-837b60d34358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:36.480327 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892654c-9d80-4827-9345-837b60d34358 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:36.480971 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892654c-9d80-4827-9345-837b60d34358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:36.482617 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892654c-9d80-4827-9345-837b60d34358 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:36.493309 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1306/2605] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:44:36.497526 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274f9b3-9279-41de-b524-d72f4c88cb44 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:36.508288 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274f9b3-9279-41de-b524-d72f4c88cb44 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:36.513636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb3789-a1a8-4c85-8c5e-61a0b86feb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:36.514286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb3789-a1a8-4c85-8c5e-61a0b86feb1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:36.514570 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb3789-a1a8-4c85-8c5e-61a0b86feb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:36.514871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb3789-a1a8-4c85-8c5e-61a0b86feb1d admin admin] PATH_INFO: `/v3/projects/4a3f0f1ab9a8447aa7e37b4f4785d7cb/users/fd0f95bb0de043369672bf968664f8f1/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:36.532180 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb3789-a1a8-4c85-8c5e-61a0b86feb1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a3f0f1ab9a8447aa7e37b4f4785d7cb, user_id=fd0f95bb0de043369672bf968664f8f1, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:36.535325 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb3789-a1a8-4c85-8c5e-61a0b86feb1d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:36.621423 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1300/2606] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:36 2022] PUT /identity/v3/projects/4a3f0f1ab9a8447aa7e37b4f4785d7cb/users/fd0f95bb0de043369672bf968664f8f1/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:36.632379 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd08de-0214-4509-b491-b596b5be3e78 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:36.632668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd08de-0214-4509-b491-b596b5be3e78 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:36.632943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd08de-0214-4509-b491-b596b5be3e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:36.659412 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb3789-a1a8-4c85-8c5e-61a0b86feb1d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:36.661623 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f1bb3789-a1a8-4c85-8c5e-61a0b86feb1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:36.670639 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-05fd08de-0214-4509-b491-b596b5be3e78 None None] MFA Rules not processed for user `fd0f95bb0de043369672bf968664f8f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:36.672172 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-05fd08de-0214-4509-b491-b596b5be3e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:36.728932 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb3789-a1a8-4c85-8c5e-61a0b86feb1d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:36.733692 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49b891-80b3-4969-a716-351b70fde913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:36.734292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49b891-80b3-4969-a716-351b70fde913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:36.734647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49b891-80b3-4969-a716-351b70fde913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:36.734832 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49b891-80b3-4969-a716-351b70fde913 admin admin] PATH_INFO: `/v3/users/d04598e263fa445f81d33287aba03cb4` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:36.735955 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49b891-80b3-4969-a716-351b70fde913 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d04598e263fa445f81d33287aba03cb4)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:36.738000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49b891-80b3-4969-a716-351b70fde913 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:36.744260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1307/2607] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:36.752608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd08de-0214-4509-b491-b596b5be3e78 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:36.754039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-05fd08de-0214-4509-b491-b596b5be3e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:36.828343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd08de-0214-4509-b491-b596b5be3e78 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:36.832613 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894aae1-e3ef-4a87-9963-5f0b17846da9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:36.833267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894aae1-e3ef-4a87-9963-5f0b17846da9 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:36.833548 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894aae1-e3ef-4a87-9963-5f0b17846da9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:36.833824 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894aae1-e3ef-4a87-9963-5f0b17846da9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:36.842442 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894aae1-e3ef-4a87-9963-5f0b17846da9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:36.843888 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894aae1-e3ef-4a87-9963-5f0b17846da9 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:36.853462 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1308/2608] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:44:36.982394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-5e49b891-80b3-4969-a716-351b70fde913 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd04598e263fa445f81d33287aba03cb4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:36.983036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-5e49b891-80b3-4969-a716-351b70fde913 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd04598e263fa445f81d33287aba03cb4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:36.987288 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:DELETE=14,keystone:SELECT=82,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:37.067460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-5e49b891-80b3-4969-a716-351b70fde913 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd04598e263fa445f81d33287aba03cb4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:37.068479 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-5e49b891-80b3-4969-a716-351b70fde913 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd04598e263fa445f81d33287aba03cb4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:37.083717 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-5e49b891-80b3-4969-a716-351b70fde913 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd04598e263fa445f81d33287aba03cb4'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:37.100241 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1301/2609] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:44:36 2022] DELETE /identity/v3/users/d04598e263fa445f81d33287aba03cb4 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:37.106129 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef117f8f-ecd9-44a2-84a1-bfab5e7f7f1d None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:37.106628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef117f8f-ecd9-44a2-84a1-bfab5e7f7f1d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:37.106729 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef117f8f-ecd9-44a2-84a1-bfab5e7f7f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:37.125150 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-ef117f8f-ecd9-44a2-84a1-bfab5e7f7f1d None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:37.126338 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-ef117f8f-ecd9-44a2-84a1-bfab5e7f7f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:37.160613 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:37.189208 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1309/2610] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:37.199805 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49b891-80b3-4969-a716-351b70fde913 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:37.201317 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5e49b891-80b3-4969-a716-351b70fde913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:37.262137 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49b891-80b3-4969-a716-351b70fde913 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:37.265868 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dc20f-3345-4f67-ab4d-8105bf485f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:37.266472 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dc20f-3345-4f67-ab4d-8105bf485f04 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:37.266746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dc20f-3345-4f67-ab4d-8105bf485f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:37.267018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dc20f-3345-4f67-ab4d-8105bf485f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:37.277356 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dc20f-3345-4f67-ab4d-8105bf485f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:37.283801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dc20f-3345-4f67-ab4d-8105bf485f04 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:37.288413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1302/2611] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:44:37.697206 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef117f8f-ecd9-44a2-84a1-bfab5e7f7f1d None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:37.698800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-ef117f8f-ecd9-44a2-84a1-bfab5e7f7f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:37.710857 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef117f8f-ecd9-44a2-84a1-bfab5e7f7f1d None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:37.715452 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb50fb7-cd8c-4801-944a-060c80569da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:37.716234 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb50fb7-cd8c-4801-944a-060c80569da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:37.716517 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb50fb7-cd8c-4801-944a-060c80569da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:37.716795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb50fb7-cd8c-4801-944a-060c80569da3 admin admin] PATH_INFO: `/v3/projects/3e018cfe2e04479d92ba517315b92c8b` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:37.717955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb50fb7-cd8c-4801-944a-060c80569da3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e018cfe2e04479d92ba517315b92c8b)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:37.722723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb50fb7-cd8c-4801-944a-060c80569da3 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:37.843986 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-cdb50fb7-cd8c-4801-944a-060c80569da3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e018cfe2e04479d92ba517315b92c8b', 'request_id': 'req-cdb50fb7-cd8c-4801-944a-060c80569da3'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:37.850062 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-cdb50fb7-cd8c-4801-944a-060c80569da3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e018cfe2e04479d92ba517315b92c8b', 'request_id': 'req-cdb50fb7-cd8c-4801-944a-060c80569da3'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:37.856172 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-cdb50fb7-cd8c-4801-944a-060c80569da3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e018cfe2e04479d92ba517315b92c8b', 'request_id': 'req-cdb50fb7-cd8c-4801-944a-060c80569da3'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:37.956658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-cdb50fb7-cd8c-4801-944a-060c80569da3 admin admin] The token cache is being invalidate because project 3e018cfe2e04479d92ba517315b92c8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:44:37.956979 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-cdb50fb7-cd8c-4801-944a-060c80569da3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:37.962526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-cdb50fb7-cd8c-4801-944a-060c80569da3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:37.962526 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1310/2612] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:44:37 2022] DELETE /identity/v3/projects/3e018cfe2e04479d92ba517315b92c8b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:38.740461 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dc20f-3345-4f67-ab4d-8105bf485f04 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:38.742060 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-470dc20f-3345-4f67-ab4d-8105bf485f04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:38.755903 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dc20f-3345-4f67-ab4d-8105bf485f04 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:38.814315 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2a05c-9d4c-4782-bc7e-f7e782636726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:38.815089 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2a05c-9d4c-4782-bc7e-f7e782636726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:38.815868 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2a05c-9d4c-4782-bc7e-f7e782636726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:38.816143 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2a05c-9d4c-4782-bc7e-f7e782636726 admin admin] PATH_INFO: `/v3/users/fc3065685ea847f9a823efd2d12674cf` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:38.817347 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2a05c-9d4c-4782-bc7e-f7e782636726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc3065685ea847f9a823efd2d12674cf)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:38.819284 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2a05c-9d4c-4782-bc7e-f7e782636726 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:39.040229 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7dd2a05c-9d4c-4782-bc7e-f7e782636726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc3065685ea847f9a823efd2d12674cf'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:39.040887 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7dd2a05c-9d4c-4782-bc7e-f7e782636726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc3065685ea847f9a823efd2d12674cf'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:39.141173 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7dd2a05c-9d4c-4782-bc7e-f7e782636726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc3065685ea847f9a823efd2d12674cf'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:39.141853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7dd2a05c-9d4c-4782-bc7e-f7e782636726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc3065685ea847f9a823efd2d12674cf'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:39.154746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-7dd2a05c-9d4c-4782-bc7e-f7e782636726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc3065685ea847f9a823efd2d12674cf'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:39.156801 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:39.168275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1303/2613] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:44:38 2022] DELETE /identity/v3/users/fc3065685ea847f9a823efd2d12674cf => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:39.172784 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce1df7-6d4e-4415-a708-c5e06a191d9d None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:39.173297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce1df7-6d4e-4415-a708-c5e06a191d9d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:39.173407 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce1df7-6d4e-4415-a708-c5e06a191d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:39.192723 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-16ce1df7-6d4e-4415-a708-c5e06a191d9d None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:39.193949 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-16ce1df7-6d4e-4415-a708-c5e06a191d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:39.258649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1311/2614] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:39.280178 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2a05c-9d4c-4782-bc7e-f7e782636726 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:39.281627 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7dd2a05c-9d4c-4782-bc7e-f7e782636726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:39.337919 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2a05c-9d4c-4782-bc7e-f7e782636726 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:39.341629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e38b9-076c-428b-bc2f-8ccee14a651b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:39.342231 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e38b9-076c-428b-bc2f-8ccee14a651b service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:39.342515 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e38b9-076c-428b-bc2f-8ccee14a651b service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:39.342791 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e38b9-076c-428b-bc2f-8ccee14a651b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:39.352223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e38b9-076c-428b-bc2f-8ccee14a651b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:39.353686 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e38b9-076c-428b-bc2f-8ccee14a651b service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:39.363932 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1304/2615] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:44:39.801778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce1df7-6d4e-4415-a708-c5e06a191d9d None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:39.803203 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-16ce1df7-6d4e-4415-a708-c5e06a191d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:39.815180 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce1df7-6d4e-4415-a708-c5e06a191d9d None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:39.820297 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc12523-e6d4-4024-ace7-a4db74e1d1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:39.820931 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc12523-e6d4-4024-ace7-a4db74e1d1bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:39.821245 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc12523-e6d4-4024-ace7-a4db74e1d1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:39.821523 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc12523-e6d4-4024-ace7-a4db74e1d1bc admin admin] PATH_INFO: `/v3/projects/8eed977aa4ec4ce4b7898d35b37b5896` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:39.822663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc12523-e6d4-4024-ace7-a4db74e1d1bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eed977aa4ec4ce4b7898d35b37b5896)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:39.824573 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc12523-e6d4-4024-ace7-a4db74e1d1bc admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:39.947916 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-0dc12523-e6d4-4024-ace7-a4db74e1d1bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eed977aa4ec4ce4b7898d35b37b5896', 'request_id': 'req-0dc12523-e6d4-4024-ace7-a4db74e1d1bc'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:39.953622 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-0dc12523-e6d4-4024-ace7-a4db74e1d1bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eed977aa4ec4ce4b7898d35b37b5896', 'request_id': 'req-0dc12523-e6d4-4024-ace7-a4db74e1d1bc'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:39.954702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:39.961204 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-0dc12523-e6d4-4024-ace7-a4db74e1d1bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eed977aa4ec4ce4b7898d35b37b5896', 'request_id': 'req-0dc12523-e6d4-4024-ace7-a4db74e1d1bc'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:40.031808 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-0dc12523-e6d4-4024-ace7-a4db74e1d1bc admin admin] The token cache is being invalidate because project 8eed977aa4ec4ce4b7898d35b37b5896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:44:40.032199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-0dc12523-e6d4-4024-ace7-a4db74e1d1bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:40.032837 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-0dc12523-e6d4-4024-ace7-a4db74e1d1bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:40.034411 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1312/2616] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:44:39 2022] DELETE /identity/v3/projects/8eed977aa4ec4ce4b7898d35b37b5896 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:40.039447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5273e-fab9-40c6-b8cb-3edd978d24ca None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:40.039971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5273e-fab9-40c6-b8cb-3edd978d24ca None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:40.039971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5273e-fab9-40c6-b8cb-3edd978d24ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:40.059315 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a1c5273e-fab9-40c6-b8cb-3edd978d24ca None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:40.060517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a1c5273e-fab9-40c6-b8cb-3edd978d24ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:40.123448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1305/2617] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:40.131889 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef69894-c879-4765-9c8c-5fef466c481b None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:40.132169 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef69894-c879-4765-9c8c-5fef466c481b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:40.132443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef69894-c879-4765-9c8c-5fef466c481b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:40.153454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-1ef69894-c879-4765-9c8c-5fef466c481b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:40.154656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-1ef69894-c879-4765-9c8c-5fef466c481b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:40.162533 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1313/2618] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:40.166721 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5273e-fab9-40c6-b8cb-3edd978d24ca None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:40.176460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5273e-fab9-40c6-b8cb-3edd978d24ca None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:40.180982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a5bf6-0267-463e-b388-70f368c683d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:40.181641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a5bf6-0267-463e-b388-70f368c683d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:40.181919 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a5bf6-0267-463e-b388-70f368c683d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:40.182190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a5bf6-0267-463e-b388-70f368c683d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:40.182836 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a5bf6-0267-463e-b388-70f368c683d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:44:40.183138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a5bf6-0267-463e-b388-70f368c683d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:40.184707 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a5bf6-0267-463e-b388-70f368c683d7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:40.196503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1306/2619] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:44:40.200629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef69894-c879-4765-9c8c-5fef466c481b None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:40.210386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef69894-c879-4765-9c8c-5fef466c481b None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:40.215029 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee41d0-cb53-4c9b-beca-9a6ff984b299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:40.215615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee41d0-cb53-4c9b-beca-9a6ff984b299 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:40.215893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee41d0-cb53-4c9b-beca-9a6ff984b299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:40.216166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee41d0-cb53-4c9b-beca-9a6ff984b299 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:40.216790 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee41d0-cb53-4c9b-beca-9a6ff984b299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:40.218730 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee41d0-cb53-4c9b-beca-9a6ff984b299 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:40.378055 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1314/2620] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:44:40 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:44:40.382023 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a5bf6-0267-463e-b388-70f368c683d7 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:40.391926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a5bf6-0267-463e-b388-70f368c683d7 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:40.464141 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d18a86-8959-428a-8379-ff20d664c620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:40.464752 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d18a86-8959-428a-8379-ff20d664c620 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:40.465036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d18a86-8959-428a-8379-ff20d664c620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:40.465348 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d18a86-8959-428a-8379-ff20d664c620 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:40.465999 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d18a86-8959-428a-8379-ff20d664c620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:40.467925 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d18a86-8959-428a-8379-ff20d664c620 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:40.581992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1307/2621] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:44:40 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:44:40.586364 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee41d0-cb53-4c9b-beca-9a6ff984b299 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:40.597336 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee41d0-cb53-4c9b-beca-9a6ff984b299 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:40.602105 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136b78d-50c8-455e-ac44-7d739fb1841e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:40.602711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136b78d-50c8-455e-ac44-7d739fb1841e admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:40.602987 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136b78d-50c8-455e-ac44-7d739fb1841e admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:40.603262 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136b78d-50c8-455e-ac44-7d739fb1841e admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:40.603928 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136b78d-50c8-455e-ac44-7d739fb1841e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:40.605555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136b78d-50c8-455e-ac44-7d739fb1841e admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:40.615263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1315/2622] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:44:40.619097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d18a86-8959-428a-8379-ff20d664c620 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:40.629236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d18a86-8959-428a-8379-ff20d664c620 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:40.633853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67d01a-2f7c-43fc-a640-d1833ca51b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:40.634436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67d01a-2f7c-43fc-a640-d1833ca51b0c admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:40.634712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67d01a-2f7c-43fc-a640-d1833ca51b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:40.634987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67d01a-2f7c-43fc-a640-d1833ca51b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:40.635617 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67d01a-2f7c-43fc-a640-d1833ca51b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:40.637217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67d01a-2f7c-43fc-a640-d1833ca51b0c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:40.642334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:40.648845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1308/2623] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:44:40.652919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136b78d-50c8-455e-ac44-7d739fb1841e admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:40.663528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136b78d-50c8-455e-ac44-7d739fb1841e admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:40.668070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75c060-08e2-4eac-83b6-49756d3844a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:40.668716 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75c060-08e2-4eac-83b6-49756d3844a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:40.668994 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75c060-08e2-4eac-83b6-49756d3844a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:40.669312 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75c060-08e2-4eac-83b6-49756d3844a7 admin admin] PATH_INFO: `/v3/projects/af021dbd14a5406ab91700845ee808ef/users/33b40edf99124598b633ff30b4e31a14/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:40.683815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75c060-08e2-4eac-83b6-49756d3844a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af021dbd14a5406ab91700845ee808ef, user_id=33b40edf99124598b633ff30b4e31a14, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:40.686568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75c060-08e2-4eac-83b6-49756d3844a7 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:40.770921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1316/2624] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:40 2022] PUT /identity/v3/projects/af021dbd14a5406ab91700845ee808ef/users/33b40edf99124598b633ff30b4e31a14/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:40.781569 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61be22d-a24e-4414-9859-2d15d366ae5a None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:40.781855 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61be22d-a24e-4414-9859-2d15d366ae5a None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:40.782133 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61be22d-a24e-4414-9859-2d15d366ae5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:40.820687 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b61be22d-a24e-4414-9859-2d15d366ae5a None None] MFA Rules not processed for user `33b40edf99124598b633ff30b4e31a14`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:40.821973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b61be22d-a24e-4414-9859-2d15d366ae5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:40.886153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1309/2625] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:40.890520 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75c060-08e2-4eac-83b6-49756d3844a7 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:40.953895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75c060-08e2-4eac-83b6-49756d3844a7 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:40.958749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b806d-07dc-4c43-a0cc-26d4a36f91a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:40.959355 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b806d-07dc-4c43-a0cc-26d4a36f91a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:40.959636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b806d-07dc-4c43-a0cc-26d4a36f91a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:40.959910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b806d-07dc-4c43-a0cc-26d4a36f91a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:40.960538 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b806d-07dc-4c43-a0cc-26d4a36f91a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:40.962509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b806d-07dc-4c43-a0cc-26d4a36f91a9 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:41.027796 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1317/2626] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:44:40 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:44:41.031792 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61be22d-a24e-4414-9859-2d15d366ae5a None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:41.043295 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61be22d-a24e-4414-9859-2d15d366ae5a None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:41.099568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d770d-b101-485a-9e0b-86f04a850441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:41.100173 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d770d-b101-485a-9e0b-86f04a850441 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:41.100446 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d770d-b101-485a-9e0b-86f04a850441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:41.100714 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d770d-b101-485a-9e0b-86f04a850441 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:41.101454 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d770d-b101-485a-9e0b-86f04a850441 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:41.103373 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d770d-b101-485a-9e0b-86f04a850441 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:41.174868 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1310/2627] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:44:41 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:44:41.178935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b806d-07dc-4c43-a0cc-26d4a36f91a9 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:41.189340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b806d-07dc-4c43-a0cc-26d4a36f91a9 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:41.193975 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb527806-0527-4b74-9b9e-c852aa7a6280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:41.194566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb527806-0527-4b74-9b9e-c852aa7a6280 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:41.194843 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb527806-0527-4b74-9b9e-c852aa7a6280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:41.195114 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb527806-0527-4b74-9b9e-c852aa7a6280 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:41.195747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb527806-0527-4b74-9b9e-c852aa7a6280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:41.197483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb527806-0527-4b74-9b9e-c852aa7a6280 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:41.208388 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1318/2628] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:44:41.212382 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d770d-b101-485a-9e0b-86f04a850441 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:41.222951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d770d-b101-485a-9e0b-86f04a850441 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:41.227528 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbde3ea-2761-4b13-b372-7a6c34d78bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:41.228162 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbde3ea-2761-4b13-b372-7a6c34d78bfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:41.228436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbde3ea-2761-4b13-b372-7a6c34d78bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:41.228708 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbde3ea-2761-4b13-b372-7a6c34d78bfb admin admin] PATH_INFO: `/v3/projects/5eb8972b0a4e4a128845a1350e9ec590/users/b5f5ce09258f4ea383040bea1bddcdb1/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:41.242899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbde3ea-2761-4b13-b372-7a6c34d78bfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eb8972b0a4e4a128845a1350e9ec590, user_id=b5f5ce09258f4ea383040bea1bddcdb1, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:41.245557 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbde3ea-2761-4b13-b372-7a6c34d78bfb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:41.336003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1311/2629] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:41 2022] PUT /identity/v3/projects/5eb8972b0a4e4a128845a1350e9ec590/users/b5f5ce09258f4ea383040bea1bddcdb1/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:41.344328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb527806-0527-4b74-9b9e-c852aa7a6280 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:41.355126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb527806-0527-4b74-9b9e-c852aa7a6280 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:41.405081 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:41.411617 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74873a3-9671-45b6-9173-b9abd229b7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:41.412217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74873a3-9671-45b6-9173-b9abd229b7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:41.412516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74873a3-9671-45b6-9173-b9abd229b7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:41.412814 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74873a3-9671-45b6-9173-b9abd229b7b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:41.413481 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74873a3-9671-45b6-9173-b9abd229b7b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:41.415067 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74873a3-9671-45b6-9173-b9abd229b7b6 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:41.424674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1319/2630] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:44:41.428829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbde3ea-2761-4b13-b372-7a6c34d78bfb admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:41.439122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbde3ea-2761-4b13-b372-7a6c34d78bfb admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:41.444268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1d768-6a60-4e6b-874b-5791dd1a4ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:41.444910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1d768-6a60-4e6b-874b-5791dd1a4ede admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:41.445231 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1d768-6a60-4e6b-874b-5791dd1a4ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:41.445507 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1d768-6a60-4e6b-874b-5791dd1a4ede admin admin] PATH_INFO: `/v3/projects/5eb8972b0a4e4a128845a1350e9ec590/users/b5f5ce09258f4ea383040bea1bddcdb1/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:41.447419 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1d768-6a60-4e6b-874b-5791dd1a4ede admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eb8972b0a4e4a128845a1350e9ec590, user_id=b5f5ce09258f4ea383040bea1bddcdb1, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:41.450087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1d768-6a60-4e6b-874b-5791dd1a4ede admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:41.460749 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1312/2631] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:41 2022] PUT /identity/v3/projects/5eb8972b0a4e4a128845a1350e9ec590/users/b5f5ce09258f4ea383040bea1bddcdb1/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:41.472193 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6bceb-efff-4dbe-bd83-c62b8095cf3f None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:41.472742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6bceb-efff-4dbe-bd83-c62b8095cf3f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:41.473115 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6bceb-efff-4dbe-bd83-c62b8095cf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:41.501649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-eeb6bceb-efff-4dbe-bd83-c62b8095cf3f None None] MFA Rules not processed for user `b5f5ce09258f4ea383040bea1bddcdb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:41.503386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-eeb6bceb-efff-4dbe-bd83-c62b8095cf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:41.570682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1320/2632] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:41.581842 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1d768-6a60-4e6b-874b-5791dd1a4ede admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:41.583509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-46a1d768-6a60-4e6b-874b-5791dd1a4ede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:41.630621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:41.642107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1d768-6a60-4e6b-874b-5791dd1a4ede admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:41.646147 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca00de-9239-45ec-a120-29c421418a7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:41.647003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca00de-9239-45ec-a120-29c421418a7a service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:41.647284 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca00de-9239-45ec-a120-29c421418a7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:41.647560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca00de-9239-45ec-a120-29c421418a7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:41.657431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca00de-9239-45ec-a120-29c421418a7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:41.661208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca00de-9239-45ec-a120-29c421418a7a service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:41.668975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1313/2633] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:44:41.898563 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6bceb-efff-4dbe-bd83-c62b8095cf3f None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:41.908138 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6bceb-efff-4dbe-bd83-c62b8095cf3f None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:41.912172 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eddce2-dc2d-441c-9e63-dcd748b86c99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:41.912787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eddce2-dc2d-441c-9e63-dcd748b86c99 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:41.913081 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eddce2-dc2d-441c-9e63-dcd748b86c99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:41.913355 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eddce2-dc2d-441c-9e63-dcd748b86c99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:41.969989 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eddce2-dc2d-441c-9e63-dcd748b86c99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:41.972052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eddce2-dc2d-441c-9e63-dcd748b86c99 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:41.981253 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1321/2634] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:44:51.439336 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca00de-9239-45ec-a120-29c421418a7a service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:51.440772 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c4ca00de-9239-45ec-a120-29c421418a7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:51.505840 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca00de-9239-45ec-a120-29c421418a7a service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:51.510576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca655d2-9ae7-48eb-97bf-772f224c5aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:51.511205 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca655d2-9ae7-48eb-97bf-772f224c5aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:51.511502 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca655d2-9ae7-48eb-97bf-772f224c5aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:51.511758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca655d2-9ae7-48eb-97bf-772f224c5aad admin admin] PATH_INFO: `/v3/users/d6f8d7e0d2d6451090c20dae6543e59d` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:51.512879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca655d2-9ae7-48eb-97bf-772f224c5aad admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6f8d7e0d2d6451090c20dae6543e59d)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:51.514812 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca655d2-9ae7-48eb-97bf-772f224c5aad admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:51.696848 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-aca655d2-9ae7-48eb-97bf-772f224c5aad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6f8d7e0d2d6451090c20dae6543e59d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:51.697530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-aca655d2-9ae7-48eb-97bf-772f224c5aad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6f8d7e0d2d6451090c20dae6543e59d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:51.758066 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-aca655d2-9ae7-48eb-97bf-772f224c5aad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6f8d7e0d2d6451090c20dae6543e59d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:51.758844 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-aca655d2-9ae7-48eb-97bf-772f224c5aad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6f8d7e0d2d6451090c20dae6543e59d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:51.772770 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-aca655d2-9ae7-48eb-97bf-772f224c5aad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6f8d7e0d2d6451090c20dae6543e59d'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:51.785675 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1314/2635] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:44:51 2022] DELETE /identity/v3/users/d6f8d7e0d2d6451090c20dae6543e59d => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:51.790187 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfb997-ae83-4829-8d86-d145eddaed1a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:51.790663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfb997-ae83-4829-8d86-d145eddaed1a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:51.790779 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfb997-ae83-4829-8d86-d145eddaed1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:51.810035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-15cfb997-ae83-4829-8d86-d145eddaed1a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:51.811240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-15cfb997-ae83-4829-8d86-d145eddaed1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:51.870724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1322/2636] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:51.880512 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca655d2-9ae7-48eb-97bf-772f224c5aad admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:51.881948 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-aca655d2-9ae7-48eb-97bf-772f224c5aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:51.938838 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca655d2-9ae7-48eb-97bf-772f224c5aad admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:51.942915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa63ed-5968-40d2-b3d3-a6408fa11120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:51.943515 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa63ed-5968-40d2-b3d3-a6408fa11120 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:51.943887 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa63ed-5968-40d2-b3d3-a6408fa11120 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:51.944074 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa63ed-5968-40d2-b3d3-a6408fa11120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:51.952962 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa63ed-5968-40d2-b3d3-a6408fa11120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:51.954415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa63ed-5968-40d2-b3d3-a6408fa11120 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:51.964296 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1315/2637] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:44:52.229148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfb997-ae83-4829-8d86-d145eddaed1a None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:52.230629 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-15cfb997-ae83-4829-8d86-d145eddaed1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:52.242419 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfb997-ae83-4829-8d86-d145eddaed1a None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:52.247218 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb60ad-e309-4a50-8891-6a265b2cef2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:52.247850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb60ad-e309-4a50-8891-6a265b2cef2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:52.248125 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb60ad-e309-4a50-8891-6a265b2cef2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:52.248396 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb60ad-e309-4a50-8891-6a265b2cef2a admin admin] PATH_INFO: `/v3/users/fd0f95bb0de043369672bf968664f8f1` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:52.249627 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb60ad-e309-4a50-8891-6a265b2cef2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd0f95bb0de043369672bf968664f8f1)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:52.251581 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb60ad-e309-4a50-8891-6a265b2cef2a admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:52.266841 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:52.378366 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa63ed-5968-40d2-b3d3-a6408fa11120 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:52.380744 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a0aa63ed-5968-40d2-b3d3-a6408fa11120 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:52.396511 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa63ed-5968-40d2-b3d3-a6408fa11120 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:52.403003 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261b587-92dc-4cda-8a7e-ab2d991dae28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:52.403641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261b587-92dc-4cda-8a7e-ab2d991dae28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:52.403924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261b587-92dc-4cda-8a7e-ab2d991dae28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:52.404201 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261b587-92dc-4cda-8a7e-ab2d991dae28 admin admin] PATH_INFO: `/v3/projects/aacbec8cbf9f4cab8157ec36571adc96` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:52.405531 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261b587-92dc-4cda-8a7e-ab2d991dae28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aacbec8cbf9f4cab8157ec36571adc96)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:52.410766 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261b587-92dc-4cda-8a7e-ab2d991dae28 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:52.410766 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:52.474578 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-c6bb60ad-e309-4a50-8891-6a265b2cef2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd0f95bb0de043369672bf968664f8f1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:52.547019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f261b587-92dc-4cda-8a7e-ab2d991dae28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aacbec8cbf9f4cab8157ec36571adc96', 'request_id': 'req-f261b587-92dc-4cda-8a7e-ab2d991dae28'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:52.553190 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f261b587-92dc-4cda-8a7e-ab2d991dae28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aacbec8cbf9f4cab8157ec36571adc96', 'request_id': 'req-f261b587-92dc-4cda-8a7e-ab2d991dae28'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:52.557612 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-c6bb60ad-e309-4a50-8891-6a265b2cef2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd0f95bb0de043369672bf968664f8f1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:52.558228 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-c6bb60ad-e309-4a50-8891-6a265b2cef2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd0f95bb0de043369672bf968664f8f1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:52.558861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-c6bb60ad-e309-4a50-8891-6a265b2cef2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd0f95bb0de043369672bf968664f8f1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:52.559362 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f261b587-92dc-4cda-8a7e-ab2d991dae28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aacbec8cbf9f4cab8157ec36571adc96', 'request_id': 'req-f261b587-92dc-4cda-8a7e-ab2d991dae28'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:52.567400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-c6bb60ad-e309-4a50-8891-6a265b2cef2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd0f95bb0de043369672bf968664f8f1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:52.585172 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1323/2638] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:44:52 2022] DELETE /identity/v3/users/fd0f95bb0de043369672bf968664f8f1 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:52.589282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203a72c-2695-4f7e-87d7-fa940d8b2e24 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:52.589565 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203a72c-2695-4f7e-87d7-fa940d8b2e24 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:52.589837 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203a72c-2695-4f7e-87d7-fa940d8b2e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:52.612248 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-8203a72c-2695-4f7e-87d7-fa940d8b2e24 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:52.612248 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-8203a72c-2695-4f7e-87d7-fa940d8b2e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:52.639171 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f261b587-92dc-4cda-8a7e-ab2d991dae28 admin admin] The token cache is being invalidate because project aacbec8cbf9f4cab8157ec36571adc96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:44:52.640667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f261b587-92dc-4cda-8a7e-ab2d991dae28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:52.640667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-f261b587-92dc-4cda-8a7e-ab2d991dae28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:52.642241 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1316/2639] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:44:52 2022] DELETE /identity/v3/projects/aacbec8cbf9f4cab8157ec36571adc96 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:52.646157 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261b587-92dc-4cda-8a7e-ab2d991dae28 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:52.647823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f261b587-92dc-4cda-8a7e-ab2d991dae28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:52.675453 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1324/2640] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:52.685505 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203a72c-2695-4f7e-87d7-fa940d8b2e24 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:52.686873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-8203a72c-2695-4f7e-87d7-fa940d8b2e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:52.713705 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261b587-92dc-4cda-8a7e-ab2d991dae28 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:52.718967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c40d12-4e27-4f1f-8ad6-3351144aabd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:52.719740 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c40d12-4e27-4f1f-8ad6-3351144aabd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:52.720016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c40d12-4e27-4f1f-8ad6-3351144aabd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:52.720286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c40d12-4e27-4f1f-8ad6-3351144aabd1 admin admin] PATH_INFO: `/v3/users/7854b1efe25046c8a091eeaffdea5b78` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:52.722123 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c40d12-4e27-4f1f-8ad6-3351144aabd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7854b1efe25046c8a091eeaffdea5b78)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:52.724138 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c40d12-4e27-4f1f-8ad6-3351144aabd1 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:52.743806 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203a72c-2695-4f7e-87d7-fa940d8b2e24 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:52.747738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7d726-4566-464b-aeba-7afce77610df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:52.748352 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7d726-4566-464b-aeba-7afce77610df service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:52.748636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7d726-4566-464b-aeba-7afce77610df service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:52.748911 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7d726-4566-464b-aeba-7afce77610df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:52.759053 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7d726-4566-464b-aeba-7afce77610df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:52.760491 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7d726-4566-464b-aeba-7afce77610df service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:52.770208 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1325/2641] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:44:52.932369 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-89c40d12-4e27-4f1f-8ad6-3351144aabd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7854b1efe25046c8a091eeaffdea5b78'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:52.933018 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-89c40d12-4e27-4f1f-8ad6-3351144aabd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7854b1efe25046c8a091eeaffdea5b78'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:52.989710 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-89c40d12-4e27-4f1f-8ad6-3351144aabd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7854b1efe25046c8a091eeaffdea5b78'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:52.990395 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-89c40d12-4e27-4f1f-8ad6-3351144aabd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7854b1efe25046c8a091eeaffdea5b78'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:53.007065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-89c40d12-4e27-4f1f-8ad6-3351144aabd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7854b1efe25046c8a091eeaffdea5b78'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:53.025880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1317/2642] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:44:52 2022] DELETE /identity/v3/users/7854b1efe25046c8a091eeaffdea5b78 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:53.217250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7d726-4566-464b-aeba-7afce77610df service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:53.219610 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-7bf7d726-4566-464b-aeba-7afce77610df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:53.235610 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7d726-4566-464b-aeba-7afce77610df service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:53.280920 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:53.307990 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d479ce-ab58-4cc4-82a4-a4f4766aff98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:53.308859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d479ce-ab58-4cc4-82a4-a4f4766aff98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:53.317738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d479ce-ab58-4cc4-82a4-a4f4766aff98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:53.317738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d479ce-ab58-4cc4-82a4-a4f4766aff98 admin admin] PATH_INFO: `/v3/projects/4a3f0f1ab9a8447aa7e37b4f4785d7cb` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:53.317738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d479ce-ab58-4cc4-82a4-a4f4766aff98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a3f0f1ab9a8447aa7e37b4f4785d7cb)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:53.317738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d479ce-ab58-4cc4-82a4-a4f4766aff98 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:53.434713 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c40d12-4e27-4f1f-8ad6-3351144aabd1 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:53.436210 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-89c40d12-4e27-4f1f-8ad6-3351144aabd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:53.447202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-07d479ce-ab58-4cc4-82a4-a4f4766aff98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a3f0f1ab9a8447aa7e37b4f4785d7cb', 'request_id': 'req-07d479ce-ab58-4cc4-82a4-a4f4766aff98'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:53.449963 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c40d12-4e27-4f1f-8ad6-3351144aabd1 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:53.456399 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:53.463657 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-07d479ce-ab58-4cc4-82a4-a4f4766aff98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a3f0f1ab9a8447aa7e37b4f4785d7cb', 'request_id': 'req-07d479ce-ab58-4cc4-82a4-a4f4766aff98'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:53.469807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-07d479ce-ab58-4cc4-82a4-a4f4766aff98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a3f0f1ab9a8447aa7e37b4f4785d7cb', 'request_id': 'req-07d479ce-ab58-4cc4-82a4-a4f4766aff98'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:53.508242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff0786-a9ed-4ea4-bdb2-2243f666b6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:53.508796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff0786-a9ed-4ea4-bdb2-2243f666b6e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:53.509103 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff0786-a9ed-4ea4-bdb2-2243f666b6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:53.509388 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff0786-a9ed-4ea4-bdb2-2243f666b6e5 admin admin] PATH_INFO: `/v3/projects/93086441bccf456ea67a3cd74a0af05f` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:53.510518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff0786-a9ed-4ea4-bdb2-2243f666b6e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93086441bccf456ea67a3cd74a0af05f)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:53.512462 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff0786-a9ed-4ea4-bdb2-2243f666b6e5 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:53.558038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-07d479ce-ab58-4cc4-82a4-a4f4766aff98 admin admin] The token cache is being invalidate because project 4a3f0f1ab9a8447aa7e37b4f4785d7cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:44:53.558331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-07d479ce-ab58-4cc4-82a4-a4f4766aff98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:53.558960 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-07d479ce-ab58-4cc4-82a4-a4f4766aff98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:53.560483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1326/2643] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:44:53 2022] DELETE /identity/v3/projects/4a3f0f1ab9a8447aa7e37b4f4785d7cb => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:53.565614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4ae9c-29e4-45fd-b5a1-eaff50af151b None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:53.565894 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4ae9c-29e4-45fd-b5a1-eaff50af151b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:53.566197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4ae9c-29e4-45fd-b5a1-eaff50af151b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:53.585865 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-56a4ae9c-29e4-45fd-b5a1-eaff50af151b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:53.590471 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-56a4ae9c-29e4-45fd-b5a1-eaff50af151b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:53.594874 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1327/2644] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:53.603441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123259c2-aeaa-4910-b1fe-ff7c4733b120 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:53.603726 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123259c2-aeaa-4910-b1fe-ff7c4733b120 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:53.604006 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123259c2-aeaa-4910-b1fe-ff7c4733b120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:53.622944 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-123259c2-aeaa-4910-b1fe-ff7c4733b120 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:53.624253 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-123259c2-aeaa-4910-b1fe-ff7c4733b120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:53.626673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b9ff0786-a9ed-4ea4-bdb2-2243f666b6e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93086441bccf456ea67a3cd74a0af05f', 'request_id': 'req-b9ff0786-a9ed-4ea4-bdb2-2243f666b6e5'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:53.632720 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b9ff0786-a9ed-4ea4-bdb2-2243f666b6e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93086441bccf456ea67a3cd74a0af05f', 'request_id': 'req-b9ff0786-a9ed-4ea4-bdb2-2243f666b6e5'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:53.638992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b9ff0786-a9ed-4ea4-bdb2-2243f666b6e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93086441bccf456ea67a3cd74a0af05f', 'request_id': 'req-b9ff0786-a9ed-4ea4-bdb2-2243f666b6e5'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:53.682626 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1328/2645] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:53.686805 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123259c2-aeaa-4910-b1fe-ff7c4733b120 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:53.696520 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123259c2-aeaa-4910-b1fe-ff7c4733b120 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:53.701295 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4434c4-64a4-4086-be7f-aed221291ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:53.702415 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4434c4-64a4-4086-be7f-aed221291ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:53.702415 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4434c4-64a4-4086-be7f-aed221291ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:53.702636 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4434c4-64a4-4086-be7f-aed221291ed2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:53.703128 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4434c4-64a4-4086-be7f-aed221291ed2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:44:53.703432 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4434c4-64a4-4086-be7f-aed221291ed2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:53.705031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4434c4-64a4-4086-be7f-aed221291ed2 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:53.717343 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1329/2646] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:44:53.720221 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b9ff0786-a9ed-4ea4-bdb2-2243f666b6e5 admin admin] The token cache is being invalidate because project 93086441bccf456ea67a3cd74a0af05f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:44:53.720221 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b9ff0786-a9ed-4ea4-bdb2-2243f666b6e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:53.721079 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-b9ff0786-a9ed-4ea4-bdb2-2243f666b6e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:53.721662 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4434c4-64a4-4086-be7f-aed221291ed2 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:53.722661 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1318/2647] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:44:53 2022] DELETE /identity/v3/projects/93086441bccf456ea67a3cd74a0af05f => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:53.723500 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-ce4434c4-64a4-4086-be7f-aed221291ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:53.727084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181cc44-4d8e-4e70-95c0-b05ed0d27a1f None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:53.727374 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181cc44-4d8e-4e70-95c0-b05ed0d27a1f None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:53.727664 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181cc44-4d8e-4e70-95c0-b05ed0d27a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:53.736674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4434c4-64a4-4086-be7f-aed221291ed2 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:53.741458 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e6fa3-3e34-43ab-8bfe-379a84c1685c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:53.742074 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e6fa3-3e34-43ab-8bfe-379a84c1685c admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:53.742348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e6fa3-3e34-43ab-8bfe-379a84c1685c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:53.742660 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e6fa3-3e34-43ab-8bfe-379a84c1685c admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:53.743355 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e6fa3-3e34-43ab-8bfe-379a84c1685c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:53.745402 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e6fa3-3e34-43ab-8bfe-379a84c1685c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:53.748106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1181cc44-4d8e-4e70-95c0-b05ed0d27a1f None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:53.749342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1181cc44-4d8e-4e70-95c0-b05ed0d27a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:53.757853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1319/2648] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:53.765931 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76d834-4809-497d-8291-986a9d216eb4 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:53.766166 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76d834-4809-497d-8291-986a9d216eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:53.766448 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76d834-4809-497d-8291-986a9d216eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:53.787491 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-0c76d834-4809-497d-8291-986a9d216eb4 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:53.789038 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0c76d834-4809-497d-8291-986a9d216eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:53.798487 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1320/2649] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:53.803474 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76d834-4809-497d-8291-986a9d216eb4 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:53.812676 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76d834-4809-497d-8291-986a9d216eb4 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:53.818103 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a6096-4d83-4087-8cfc-45b74c28dd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:53.818705 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a6096-4d83-4087-8cfc-45b74c28dd46 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:53.819036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a6096-4d83-4087-8cfc-45b74c28dd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:53.819319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a6096-4d83-4087-8cfc-45b74c28dd46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:53.819974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a6096-4d83-4087-8cfc-45b74c28dd46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:44:53.820279 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a6096-4d83-4087-8cfc-45b74c28dd46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:53.821904 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a6096-4d83-4087-8cfc-45b74c28dd46 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:53.833674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1321/2650] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:44:53.837966 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a6096-4d83-4087-8cfc-45b74c28dd46 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:53.847357 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a6096-4d83-4087-8cfc-45b74c28dd46 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:53.851289 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1330/2651] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:44:53 2022] POST /identity/v3/projects => generated 361 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:44:53.853217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647cf48-9fa9-4492-a443-15caa4c3875a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:53.853217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647cf48-9fa9-4492-a443-15caa4c3875a admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:53.853217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647cf48-9fa9-4492-a443-15caa4c3875a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:53.853511 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647cf48-9fa9-4492-a443-15caa4c3875a admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:53.854009 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647cf48-9fa9-4492-a443-15caa4c3875a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:53.856380 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647cf48-9fa9-4492-a443-15caa4c3875a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:53.856513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e6fa3-3e34-43ab-8bfe-379a84c1685c admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:53.918250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e6fa3-3e34-43ab-8bfe-379a84c1685c admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:53.923076 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bbea0-fdad-4e1f-8f87-470dd127c7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:53.923682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bbea0-fdad-4e1f-8f87-470dd127c7f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:53.924794 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bbea0-fdad-4e1f-8f87-470dd127c7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:53.925119 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bbea0-fdad-4e1f-8f87-470dd127c7f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:53.925757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39bbea0-fdad-4e1f-8f87-470dd127c7f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:53.928170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39bbea0-fdad-4e1f-8f87-470dd127c7f0 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:53.932819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:53.965241 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1322/2652] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:44:53 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:44:53.969144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647cf48-9fa9-4492-a443-15caa4c3875a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:54.029321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647cf48-9fa9-4492-a443-15caa4c3875a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:54.033874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb06f6-8403-4860-919d-3b5ea640b27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:54.034484 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb06f6-8403-4860-919d-3b5ea640b27c admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:54.034755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb06f6-8403-4860-919d-3b5ea640b27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:54.035021 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb06f6-8403-4860-919d-3b5ea640b27c admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:54.035637 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb06f6-8403-4860-919d-3b5ea640b27c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:54.037575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb06f6-8403-4860-919d-3b5ea640b27c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:54.053314 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1331/2653] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:44:53 2022] POST /identity/v3/users => generated 281 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:44:54.054667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bbea0-fdad-4e1f-8f87-470dd127c7f0 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:54.064553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bbea0-fdad-4e1f-8f87-470dd127c7f0 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:54.069941 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1db5fc-0738-4e43-8df4-2634ac5d8c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:54.070528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1db5fc-0738-4e43-8df4-2634ac5d8c54 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:54.070811 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1db5fc-0738-4e43-8df4-2634ac5d8c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:54.071373 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1db5fc-0738-4e43-8df4-2634ac5d8c54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:54.072673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1db5fc-0738-4e43-8df4-2634ac5d8c54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:54.074341 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1db5fc-0738-4e43-8df4-2634ac5d8c54 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:54.084562 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1332/2654] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:44:54.088347 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1db5fc-0738-4e43-8df4-2634ac5d8c54 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:54.097569 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1db5fc-0738-4e43-8df4-2634ac5d8c54 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:54.102132 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0dc5b-a99e-4fcf-a601-45840d7ce01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:54.102686 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0dc5b-a99e-4fcf-a601-45840d7ce01b admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:54.102963 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0dc5b-a99e-4fcf-a601-45840d7ce01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:54.103231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0dc5b-a99e-4fcf-a601-45840d7ce01b admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:54.103837 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0dc5b-a99e-4fcf-a601-45840d7ce01b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:54.105415 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0dc5b-a99e-4fcf-a601-45840d7ce01b admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:54.117106 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1333/2655] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:44:54.120892 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0dc5b-a99e-4fcf-a601-45840d7ce01b admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:54.131072 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0dc5b-a99e-4fcf-a601-45840d7ce01b admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:54.136899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1323/2656] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:44:53 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:44:54.137039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666e7f9-7c1a-4b13-99c1-94f370911aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:54.137039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666e7f9-7c1a-4b13-99c1-94f370911aad admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:54.137039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666e7f9-7c1a-4b13-99c1-94f370911aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:54.137039 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666e7f9-7c1a-4b13-99c1-94f370911aad admin admin] PATH_INFO: `/v3/projects/df1b245001de4ac2a69994fb2263e8b9/users/bfbbf26c0274419cb4e3f8ebda862ba2/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:54.142654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb06f6-8403-4860-919d-3b5ea640b27c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:54.152107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666e7f9-7c1a-4b13-99c1-94f370911aad admin admin] RBAC: Authorizing `identity:create_grant(project_id=df1b245001de4ac2a69994fb2263e8b9, user_id=bfbbf26c0274419cb4e3f8ebda862ba2, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:54.152574 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb06f6-8403-4860-919d-3b5ea640b27c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:54.154827 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666e7f9-7c1a-4b13-99c1-94f370911aad admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:54.157277 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69358e18-cae4-4a17-9ccc-988ac00cab29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:54.157878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69358e18-cae4-4a17-9ccc-988ac00cab29 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:54.158150 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69358e18-cae4-4a17-9ccc-988ac00cab29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:54.162735 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69358e18-cae4-4a17-9ccc-988ac00cab29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:54.162735 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69358e18-cae4-4a17-9ccc-988ac00cab29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:54.162735 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69358e18-cae4-4a17-9ccc-988ac00cab29 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:54.172199 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1324/2657] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:44:54.176440 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69358e18-cae4-4a17-9ccc-988ac00cab29 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:54.186724 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69358e18-cae4-4a17-9ccc-988ac00cab29 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:54.191401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bc5f1-7240-4cbd-8768-0e555ef43f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:54.192097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bc5f1-7240-4cbd-8768-0e555ef43f56 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:54.192374 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bc5f1-7240-4cbd-8768-0e555ef43f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:54.192652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bc5f1-7240-4cbd-8768-0e555ef43f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:54.193318 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5bc5f1-7240-4cbd-8768-0e555ef43f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:54.194882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5bc5f1-7240-4cbd-8768-0e555ef43f56 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:54.214626 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1325/2658] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:44:54.215394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bc5f1-7240-4cbd-8768-0e555ef43f56 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:54.225790 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bc5f1-7240-4cbd-8768-0e555ef43f56 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:54.230980 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37ca38-aec8-4af5-ab4c-c0ed26587040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:54.231623 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37ca38-aec8-4af5-ab4c-c0ed26587040 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:54.231910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37ca38-aec8-4af5-ab4c-c0ed26587040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:54.232391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37ca38-aec8-4af5-ab4c-c0ed26587040 admin admin] PATH_INFO: `/v3/projects/2bab654cebb846d2b41b6da2183cd33b/users/2b48df56e7804141bd9d81a936ede89b/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:54.233856 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1334/2659] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:54 2022] PUT /identity/v3/projects/df1b245001de4ac2a69994fb2263e8b9/users/bfbbf26c0274419cb4e3f8ebda862ba2/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:54.247135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00a005-0901-43c7-9342-449a4a06e8a1 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:54.247396 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00a005-0901-43c7-9342-449a4a06e8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:54.247676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00a005-0901-43c7-9342-449a4a06e8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:54.248397 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37ca38-aec8-4af5-ab4c-c0ed26587040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bab654cebb846d2b41b6da2183cd33b, user_id=2b48df56e7804141bd9d81a936ede89b, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:54.252068 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37ca38-aec8-4af5-ab4c-c0ed26587040 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:54.262693 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:54.280580 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-8f00a005-0901-43c7-9342-449a4a06e8a1 None None] MFA Rules not processed for user `bfbbf26c0274419cb4e3f8ebda862ba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:54.280951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-8f00a005-0901-43c7-9342-449a4a06e8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:54.350860 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1335/2660] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:54.368698 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1326/2661] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:54 2022] PUT /identity/v3/projects/2bab654cebb846d2b41b6da2183cd33b/users/2b48df56e7804141bd9d81a936ede89b/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:54.389369 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0cf75-d687-495f-aca1-f1c292db1dbb None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:54.389886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0cf75-d687-495f-aca1-f1c292db1dbb None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:54.390169 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0cf75-d687-495f-aca1-f1c292db1dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:54.396089 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37ca38-aec8-4af5-ab4c-c0ed26587040 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:54.398717 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-7a37ca38-aec8-4af5-ab4c-c0ed26587040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:54.423457 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-e7d0cf75-d687-495f-aca1-f1c292db1dbb None None] MFA Rules not processed for user `2b48df56e7804141bd9d81a936ede89b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:54.423457 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e7d0cf75-d687-495f-aca1-f1c292db1dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:54.458116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37ca38-aec8-4af5-ab4c-c0ed26587040 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:54.462048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5ce87-1c07-41ba-9df4-cbecc7525020 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:54.464934 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5ce87-1c07-41ba-9df4-cbecc7525020 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:54.465250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5ce87-1c07-41ba-9df4-cbecc7525020 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:54.465523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5ce87-1c07-41ba-9df4-cbecc7525020 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:54.504878 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1336/2662] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:54.508967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0cf75-d687-495f-aca1-f1c292db1dbb None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:54.524574 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5ce87-1c07-41ba-9df4-cbecc7525020 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:54.526208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5ce87-1c07-41ba-9df4-cbecc7525020 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:54.535457 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1327/2663] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:44:54.558270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5ce87-1c07-41ba-9df4-cbecc7525020 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:54.576585 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0cf75-d687-495f-aca1-f1c292db1dbb None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:54.581634 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd66071-0975-4ab5-b179-1a870274d7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:54.582245 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd66071-0975-4ab5-b179-1a870274d7e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:54.582538 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd66071-0975-4ab5-b179-1a870274d7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:54.582910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd66071-0975-4ab5-b179-1a870274d7e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:54.583545 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd66071-0975-4ab5-b179-1a870274d7e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:54.585528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd66071-0975-4ab5-b179-1a870274d7e9 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:54.620043 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5ce87-1c07-41ba-9df4-cbecc7525020 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:54.624655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce32ccb-1674-4537-88fd-e86ec5670956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:54.625313 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce32ccb-1674-4537-88fd-e86ec5670956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:54.630108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce32ccb-1674-4537-88fd-e86ec5670956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:54.630108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce32ccb-1674-4537-88fd-e86ec5670956 admin admin] PATH_INFO: `/v3/users/bfbbf26c0274419cb4e3f8ebda862ba2` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:54.630108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce32ccb-1674-4537-88fd-e86ec5670956 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfbbf26c0274419cb4e3f8ebda862ba2)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:54.630108 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce32ccb-1674-4537-88fd-e86ec5670956 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:54.718804 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1337/2664] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:44:54 2022] POST /identity/v3/projects => generated 379 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:44:54.719483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd66071-0975-4ab5-b179-1a870274d7e9 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:54.785850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd66071-0975-4ab5-b179-1a870274d7e9 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:54.790581 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cccb0-d7d2-47c5-a134-ef74525478a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:54.791244 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cccb0-d7d2-47c5-a134-ef74525478a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:54.791727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cccb0-d7d2-47c5-a134-ef74525478a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:54.792180 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cccb0-d7d2-47c5-a134-ef74525478a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:54.793980 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:54.796664 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cccb0-d7d2-47c5-a134-ef74525478a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:54.796664 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cccb0-d7d2-47c5-a134-ef74525478a3 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:54.837091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fce32ccb-1674-4537-88fd-e86ec5670956 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfbbf26c0274419cb4e3f8ebda862ba2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:54.837882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fce32ccb-1674-4537-88fd-e86ec5670956 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfbbf26c0274419cb4e3f8ebda862ba2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:54.912404 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fce32ccb-1674-4537-88fd-e86ec5670956 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfbbf26c0274419cb4e3f8ebda862ba2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:54.913030 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fce32ccb-1674-4537-88fd-e86ec5670956 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfbbf26c0274419cb4e3f8ebda862ba2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:54.921519 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1338/2665] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:44:54 2022] POST /identity/v3/users => generated 290 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:44:54.922575 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cccb0-d7d2-47c5-a134-ef74525478a3 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:54.924827 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-a72cccb0-d7d2-47c5-a134-ef74525478a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:54.926618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-fce32ccb-1674-4537-88fd-e86ec5670956 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfbbf26c0274419cb4e3f8ebda862ba2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:54.936671 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cccb0-d7d2-47c5-a134-ef74525478a3 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:54.939176 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1328/2666] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:44:54 2022] DELETE /identity/v3/users/bfbbf26c0274419cb4e3f8ebda862ba2 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:54.947658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604c60d-20cd-4c8e-8aea-fdc363ff8366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:54.947658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604c60d-20cd-4c8e-8aea-fdc363ff8366 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:54.947658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604c60d-20cd-4c8e-8aea-fdc363ff8366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:54.947658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604c60d-20cd-4c8e-8aea-fdc363ff8366 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:54.948068 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604c60d-20cd-4c8e-8aea-fdc363ff8366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:54.948068 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604c60d-20cd-4c8e-8aea-fdc363ff8366 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:54.950391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e47893-b37f-4168-acb4-d1e5404ed48e None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:54.950674 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e47893-b37f-4168-acb4-d1e5404ed48e None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:54.950950 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e47893-b37f-4168-acb4-d1e5404ed48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:54.959691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1339/2667] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:44:54.960812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604c60d-20cd-4c8e-8aea-fdc363ff8366 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:54.964467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:54.970745 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604c60d-20cd-4c8e-8aea-fdc363ff8366 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:54.974954 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-38e47893-b37f-4168-acb4-d1e5404ed48e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:54.976166 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-38e47893-b37f-4168-acb4-d1e5404ed48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:55.027879 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b1927-a021-4e68-84fd-b80e74681765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:55.028495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b1927-a021-4e68-84fd-b80e74681765 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:55.028775 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b1927-a021-4e68-84fd-b80e74681765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:55.029128 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b1927-a021-4e68-84fd-b80e74681765 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:55.029766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b1927-a021-4e68-84fd-b80e74681765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:55.031435 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b1927-a021-4e68-84fd-b80e74681765 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:55.035180 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1329/2668] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:55.042231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1340/2669] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:44:55.046212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e47893-b37f-4168-acb4-d1e5404ed48e None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:55.052271 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-38e47893-b37f-4168-acb4-d1e5404ed48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:55.053248 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b1927-a021-4e68-84fd-b80e74681765 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:55.063257 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b1927-a021-4e68-84fd-b80e74681765 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:55.068830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24374f96-ec5f-4929-a527-b07d949f2bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:55.069522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24374f96-ec5f-4929-a527-b07d949f2bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:55.069817 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24374f96-ec5f-4929-a527-b07d949f2bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:55.070096 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24374f96-ec5f-4929-a527-b07d949f2bbe admin admin] PATH_INFO: `/v3/projects/5f2666c268ac46bfb6046ab379ee7367/users/7e65e3808cda48b4a288db83233c17ef/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:55.085759 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24374f96-ec5f-4929-a527-b07d949f2bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f2666c268ac46bfb6046ab379ee7367, user_id=7e65e3808cda48b4a288db83233c17ef, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:55.088667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24374f96-ec5f-4929-a527-b07d949f2bbe admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:55.101520 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:55.107589 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e47893-b37f-4168-acb4-d1e5404ed48e None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:55.112416 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3a573-59cd-42a2-b469-3ef9afef5857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:55.112567 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3a573-59cd-42a2-b469-3ef9afef5857 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:55.112784 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3a573-59cd-42a2-b469-3ef9afef5857 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:55.113083 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3a573-59cd-42a2-b469-3ef9afef5857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:55.121868 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3a573-59cd-42a2-b469-3ef9afef5857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:55.123273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3a573-59cd-42a2-b469-3ef9afef5857 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:55.132648 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1330/2670] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:44:55.154152 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1341/2671] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:55 2022] PUT /identity/v3/projects/5f2666c268ac46bfb6046ab379ee7367/users/7e65e3808cda48b4a288db83233c17ef/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:55.166192 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28973d-4689-4be2-ab14-7f87641ad2fa None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:55.166622 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28973d-4689-4be2-ab14-7f87641ad2fa None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:55.167149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28973d-4689-4be2-ab14-7f87641ad2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:55.190718 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:55.200977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4d28973d-4689-4be2-ab14-7f87641ad2fa None None] MFA Rules not processed for user `7e65e3808cda48b4a288db83233c17ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:55.202772 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4d28973d-4689-4be2-ab14-7f87641ad2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:55.273370 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1331/2672] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:55.277973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24374f96-ec5f-4929-a527-b07d949f2bbe admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:55.341789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24374f96-ec5f-4929-a527-b07d949f2bbe admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:55.347314 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c3c20-4829-4d2a-947d-60fa692e15c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:55.347929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c3c20-4829-4d2a-947d-60fa692e15c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:55.348212 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c3c20-4829-4d2a-947d-60fa692e15c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:55.348488 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c3c20-4829-4d2a-947d-60fa692e15c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:55.349491 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c3c20-4829-4d2a-947d-60fa692e15c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:55.351201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c3c20-4829-4d2a-947d-60fa692e15c9 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:55.454172 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1342/2673] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:44:55 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:44:55.455750 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28973d-4689-4be2-ab14-7f87641ad2fa None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:55.466834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28973d-4689-4be2-ab14-7f87641ad2fa None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:55.523705 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a4ba4-3b6a-4d97-a4f9-1d3cbb2c50b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:55.524313 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a4ba4-3b6a-4d97-a4f9-1d3cbb2c50b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:55.524591 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a4ba4-3b6a-4d97-a4f9-1d3cbb2c50b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:55.524865 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a4ba4-3b6a-4d97-a4f9-1d3cbb2c50b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:55.525540 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a4ba4-3b6a-4d97-a4f9-1d3cbb2c50b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:55.527499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a4ba4-3b6a-4d97-a4f9-1d3cbb2c50b0 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:55.572942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c3c20-4829-4d2a-947d-60fa692e15c9 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:55.575910 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e65c3c20-4829-4d2a-947d-60fa692e15c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:55.589805 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c3c20-4829-4d2a-947d-60fa692e15c9 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:55.596390 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acc85d-9495-4400-abb4-bced9c1c3da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:55.596390 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acc85d-9495-4400-abb4-bced9c1c3da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:55.596390 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acc85d-9495-4400-abb4-bced9c1c3da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:55.596733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acc85d-9495-4400-abb4-bced9c1c3da2 admin admin] PATH_INFO: `/v3/projects/df1b245001de4ac2a69994fb2263e8b9` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:55.598023 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acc85d-9495-4400-abb4-bced9c1c3da2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df1b245001de4ac2a69994fb2263e8b9)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:55.599933 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acc85d-9495-4400-abb4-bced9c1c3da2 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:55.649390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1332/2674] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:44:55 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:44:55.653335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a4ba4-3b6a-4d97-a4f9-1d3cbb2c50b0 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:55.663071 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a4ba4-3b6a-4d97-a4f9-1d3cbb2c50b0 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:55.667627 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89290f03-66bc-4104-82b1-d7b0f58dfcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:55.668192 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89290f03-66bc-4104-82b1-d7b0f58dfcb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:55.668465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89290f03-66bc-4104-82b1-d7b0f58dfcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:55.668734 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89290f03-66bc-4104-82b1-d7b0f58dfcb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:55.669408 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89290f03-66bc-4104-82b1-d7b0f58dfcb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:55.670961 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89290f03-66bc-4104-82b1-d7b0f58dfcb1 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:55.681304 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1333/2675] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:44:55.685161 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89290f03-66bc-4104-82b1-d7b0f58dfcb1 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:55.694871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89290f03-66bc-4104-82b1-d7b0f58dfcb1 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:55.699406 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b246a3-9305-477c-b55f-8ce04411a47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:55.699997 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b246a3-9305-477c-b55f-8ce04411a47e admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:55.700271 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b246a3-9305-477c-b55f-8ce04411a47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:55.700604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b246a3-9305-477c-b55f-8ce04411a47e admin admin] PATH_INFO: `/v3/projects/7a436a13d2724a15bdd068af53880ae4/users/ecce0ec6e316455f85ab72753c023813/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:55.714972 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b246a3-9305-477c-b55f-8ce04411a47e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a436a13d2724a15bdd068af53880ae4, user_id=ecce0ec6e316455f85ab72753c023813, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:55.717710 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b246a3-9305-477c-b55f-8ce04411a47e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:55.793955 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1334/2676] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:55 2022] PUT /identity/v3/projects/7a436a13d2724a15bdd068af53880ae4/users/ecce0ec6e316455f85ab72753c023813/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:55.804754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-81acc85d-9495-4400-abb4-bced9c1c3da2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df1b245001de4ac2a69994fb2263e8b9', 'request_id': 'req-81acc85d-9495-4400-abb4-bced9c1c3da2'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:55.807199 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b246a3-9305-477c-b55f-8ce04411a47e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:55.816489 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-81acc85d-9495-4400-abb4-bced9c1c3da2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df1b245001de4ac2a69994fb2263e8b9', 'request_id': 'req-81acc85d-9495-4400-abb4-bced9c1c3da2'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:55.829182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-81acc85d-9495-4400-abb4-bced9c1c3da2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df1b245001de4ac2a69994fb2263e8b9', 'request_id': 'req-81acc85d-9495-4400-abb4-bced9c1c3da2'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:55.871852 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:55.878589 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b246a3-9305-477c-b55f-8ce04411a47e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:55.886824 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-81acc85d-9495-4400-abb4-bced9c1c3da2 admin admin] The token cache is being invalidate because project df1b245001de4ac2a69994fb2263e8b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:44:55.887126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-81acc85d-9495-4400-abb4-bced9c1c3da2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:55.887859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-81acc85d-9495-4400-abb4-bced9c1c3da2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:44:55.889440 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1343/2677] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:44:55 2022] DELETE /identity/v3/projects/df1b245001de4ac2a69994fb2263e8b9 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:55.894933 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9e1b0-bd95-4167-8909-58ee6739cca0 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:55.895219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9e1b0-bd95-4167-8909-58ee6739cca0 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:55.895499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9e1b0-bd95-4167-8909-58ee6739cca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:55.920068 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-7db9e1b0-bd95-4167-8909-58ee6739cca0 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:55.921509 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-7db9e1b0-bd95-4167-8909-58ee6739cca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:55.946611 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6670ba-7041-4762-a902-afdf70472d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:55.947222 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6670ba-7041-4762-a902-afdf70472d4a admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:55.947503 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6670ba-7041-4762-a902-afdf70472d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:55.947843 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6670ba-7041-4762-a902-afdf70472d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:55.948485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6670ba-7041-4762-a902-afdf70472d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:55.948790 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:55.950104 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6670ba-7041-4762-a902-afdf70472d4a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:55.960378 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1335/2678] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:44:55.964427 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6670ba-7041-4762-a902-afdf70472d4a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:55.966269 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0e6670ba-7041-4762-a902-afdf70472d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:55.978358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6670ba-7041-4762-a902-afdf70472d4a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:55.983068 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c2d4c-df6f-4cf6-ac5c-78cf5a95660b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:55.983721 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c2d4c-df6f-4cf6-ac5c-78cf5a95660b admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:55.983997 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c2d4c-df6f-4cf6-ac5c-78cf5a95660b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:55.984272 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c2d4c-df6f-4cf6-ac5c-78cf5a95660b admin admin] PATH_INFO: `/v3/projects/7a436a13d2724a15bdd068af53880ae4/users/ecce0ec6e316455f85ab72753c023813/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:55.984906 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1344/2679] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:55.986422 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c2d4c-df6f-4cf6-ac5c-78cf5a95660b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a436a13d2724a15bdd068af53880ae4, user_id=ecce0ec6e316455f85ab72753c023813, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:55.989328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c2d4c-df6f-4cf6-ac5c-78cf5a95660b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:55.995231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162792e-41ca-4137-b2b5-c206de774948 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:55.995231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162792e-41ca-4137-b2b5-c206de774948 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:55.995231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162792e-41ca-4137-b2b5-c206de774948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:56.017206 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1336/2680] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:55 2022] PUT /identity/v3/projects/7a436a13d2724a15bdd068af53880ae4/users/ecce0ec6e316455f85ab72753c023813/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:44:56.017333 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-6162792e-41ca-4137-b2b5-c206de774948 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:56.017333 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-6162792e-41ca-4137-b2b5-c206de774948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:56.026637 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bf7fa-9306-404f-b3cb-cd7041b1dbc6 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:56.026925 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bf7fa-9306-404f-b3cb-cd7041b1dbc6 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:56.027204 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bf7fa-9306-404f-b3cb-cd7041b1dbc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:56.059843 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-8b1bf7fa-9306-404f-b3cb-cd7041b1dbc6 None None] MFA Rules not processed for user `ecce0ec6e316455f85ab72753c023813`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:56.080281 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8b1bf7fa-9306-404f-b3cb-cd7041b1dbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:56.092951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1345/2681] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:56.097116 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162792e-41ca-4137-b2b5-c206de774948 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:56.107263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162792e-41ca-4137-b2b5-c206de774948 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:56.122604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d7999-1146-429d-af8f-c2b40827734e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:56.123235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d7999-1146-429d-af8f-c2b40827734e admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:56.123519 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d7999-1146-429d-af8f-c2b40827734e admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:56.123820 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d7999-1146-429d-af8f-c2b40827734e admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:56.124477 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d7999-1146-429d-af8f-c2b40827734e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:44:56.124802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d7999-1146-429d-af8f-c2b40827734e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:56.126672 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d7999-1146-429d-af8f-c2b40827734e admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:56.139653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1346/2682] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:44:56.142736 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1337/2683] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:44:56.145933 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bf7fa-9306-404f-b3cb-cd7041b1dbc6 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:56.155616 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d7999-1146-429d-af8f-c2b40827734e admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:56.157708 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-532d7999-1146-429d-af8f-c2b40827734e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:56.160495 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bf7fa-9306-404f-b3cb-cd7041b1dbc6 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:56.165621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8eddf5-05fd-4c6b-ab9b-db69f4694839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:56.166309 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8eddf5-05fd-4c6b-ab9b-db69f4694839 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:56.166592 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8eddf5-05fd-4c6b-ab9b-db69f4694839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:56.166870 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8eddf5-05fd-4c6b-ab9b-db69f4694839 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:56.167512 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8eddf5-05fd-4c6b-ab9b-db69f4694839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:56.169457 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8eddf5-05fd-4c6b-ab9b-db69f4694839 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:56.217305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d7999-1146-429d-af8f-c2b40827734e admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:56.221388 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3160de2-c2a8-40f2-ae5f-450f83041e2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:56.222034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3160de2-c2a8-40f2-ae5f-450f83041e2a service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:56.222319 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3160de2-c2a8-40f2-ae5f-450f83041e2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:56.222593 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3160de2-c2a8-40f2-ae5f-450f83041e2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:56.232980 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3160de2-c2a8-40f2-ae5f-450f83041e2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:56.234541 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3160de2-c2a8-40f2-ae5f-450f83041e2a service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:56.244683 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1347/2684] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:44:56.291540 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1338/2685] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:44:56 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:44:56.295790 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3160de2-c2a8-40f2-ae5f-450f83041e2a service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:56.307093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3160de2-c2a8-40f2-ae5f-450f83041e2a service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:56.364639 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91763e6-8aff-47dc-af2a-711360ce5c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:56.365198 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91763e6-8aff-47dc-af2a-711360ce5c91 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:56.365479 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91763e6-8aff-47dc-af2a-711360ce5c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:56.365759 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91763e6-8aff-47dc-af2a-711360ce5c91 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:56.366387 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91763e6-8aff-47dc-af2a-711360ce5c91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:56.368455 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91763e6-8aff-47dc-af2a-711360ce5c91 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:56.380141 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=100 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:56.391091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8eddf5-05fd-4c6b-ab9b-db69f4694839 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:56.392892 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2b8eddf5-05fd-4c6b-ab9b-db69f4694839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:56.446150 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:44:56.449858 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8eddf5-05fd-4c6b-ab9b-db69f4694839 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:56.453484 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21813d6-9802-42c4-aded-1c640350e6d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:56.454129 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21813d6-9802-42c4-aded-1c640350e6d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:56.454413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21813d6-9802-42c4-aded-1c640350e6d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:56.454679 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21813d6-9802-42c4-aded-1c640350e6d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:56.456502 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e21813d6-9802-42c4-aded-1c640350e6d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:56.499200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1348/2686] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:44:56 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:44:56.500219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91763e6-8aff-47dc-af2a-711360ce5c91 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:56.510048 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91763e6-8aff-47dc-af2a-711360ce5c91 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:56.514649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129770b-72ee-4370-ae15-6f465daee625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:56.515238 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129770b-72ee-4370-ae15-6f465daee625 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:56.515512 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129770b-72ee-4370-ae15-6f465daee625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:56.515788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129770b-72ee-4370-ae15-6f465daee625 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:56.516424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129770b-72ee-4370-ae15-6f465daee625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:56.518147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129770b-72ee-4370-ae15-6f465daee625 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:56.522335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21813d6-9802-42c4-aded-1c640350e6d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:56.522335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21813d6-9802-42c4-aded-1c640350e6d3 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:56.531412 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1349/2687] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:44:56.531602 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1339/2688] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:44:56.534289 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129770b-72ee-4370-ae15-6f465daee625 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:56.544055 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129770b-72ee-4370-ae15-6f465daee625 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:56.549003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23319e-2004-4233-a497-b63c8c6170a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:56.550138 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23319e-2004-4233-a497-b63c8c6170a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:56.550418 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23319e-2004-4233-a497-b63c8c6170a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:56.550686 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23319e-2004-4233-a497-b63c8c6170a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:56.552120 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23319e-2004-4233-a497-b63c8c6170a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:56.553008 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23319e-2004-4233-a497-b63c8c6170a2 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:56.564791 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1350/2689] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:44:56.569342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21813d6-9802-42c4-aded-1c640350e6d3 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:56.580073 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21813d6-9802-42c4-aded-1c640350e6d3 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:56.584659 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a637be-a483-4a38-9d92-f175c2a0875a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:56.585349 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a637be-a483-4a38-9d92-f175c2a0875a admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:56.585696 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a637be-a483-4a38-9d92-f175c2a0875a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:56.585971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a637be-a483-4a38-9d92-f175c2a0875a admin admin] PATH_INFO: `/v3/projects/d20926a91920444ca93d2780c90363b5/users/6b2ba314d9d44033ab542dec3a0aeff7/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:56.601695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a637be-a483-4a38-9d92-f175c2a0875a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d20926a91920444ca93d2780c90363b5, user_id=6b2ba314d9d44033ab542dec3a0aeff7, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:56.609501 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a637be-a483-4a38-9d92-f175c2a0875a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:56.734700 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1340/2690] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:44:56 2022] PUT /identity/v3/projects/d20926a91920444ca93d2780c90363b5/users/6b2ba314d9d44033ab542dec3a0aeff7/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:44:56.803221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2fabb-ba7a-49e3-89a8-e693817b9bd7 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:56.803506 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2fabb-ba7a-49e3-89a8-e693817b9bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:56.803784 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2fabb-ba7a-49e3-89a8-e693817b9bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:56.832998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-79a2fabb-ba7a-49e3-89a8-e693817b9bd7 None None] MFA Rules not processed for user `6b2ba314d9d44033ab542dec3a0aeff7`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:44:56.834254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-79a2fabb-ba7a-49e3-89a8-e693817b9bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:44:56.898032 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1351/2691] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:44:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:44:56.909992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a637be-a483-4a38-9d92-f175c2a0875a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:44:56.964078 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a637be-a483-4a38-9d92-f175c2a0875a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:44:56.967758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460936f9-c4c5-46fc-8b6b-66755c10e6d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:44:56.968407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460936f9-c4c5-46fc-8b6b-66755c10e6d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:44:56.968690 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460936f9-c4c5-46fc-8b6b-66755c10e6d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:44:56.968966 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460936f9-c4c5-46fc-8b6b-66755c10e6d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:44:56.978044 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460936f9-c4c5-46fc-8b6b-66755c10e6d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:44:56.979458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460936f9-c4c5-46fc-8b6b-66755c10e6d0 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:44:56.988255 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1341/2692] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:44:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:45:03.944690 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2fabb-ba7a-49e3-89a8-e693817b9bd7 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:04.007128 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2fabb-ba7a-49e3-89a8-e693817b9bd7 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:04.012381 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3a39e-a538-4d92-941d-0554cfe8d1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:04.013010 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3a39e-a538-4d92-941d-0554cfe8d1e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:04.013500 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3a39e-a538-4d92-941d-0554cfe8d1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:04.013787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3a39e-a538-4d92-941d-0554cfe8d1e1 admin admin] PATH_INFO: `/v3/users/2b48df56e7804141bd9d81a936ede89b` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:04.014962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3a39e-a538-4d92-941d-0554cfe8d1e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b48df56e7804141bd9d81a936ede89b)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:04.017102 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3a39e-a538-4d92-941d-0554cfe8d1e1 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:04.218234 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-94b3a39e-a538-4d92-941d-0554cfe8d1e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b48df56e7804141bd9d81a936ede89b'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:04.301658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-94b3a39e-a538-4d92-941d-0554cfe8d1e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b48df56e7804141bd9d81a936ede89b'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:04.302372 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-94b3a39e-a538-4d92-941d-0554cfe8d1e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b48df56e7804141bd9d81a936ede89b'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:04.302991 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-94b3a39e-a538-4d92-941d-0554cfe8d1e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b48df56e7804141bd9d81a936ede89b'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:04.308287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:04.311292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-94b3a39e-a538-4d92-941d-0554cfe8d1e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b48df56e7804141bd9d81a936ede89b'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:04.328935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1352/2693] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:45:03 2022] DELETE /identity/v3/users/2b48df56e7804141bd9d81a936ede89b => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:04.333593 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30b2b2-aa2e-4594-9443-7478733551de None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:04.337092 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30b2b2-aa2e-4594-9443-7478733551de None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:04.337092 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30b2b2-aa2e-4594-9443-7478733551de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:04.352785 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-cb30b2b2-aa2e-4594-9443-7478733551de None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:04.354142 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cb30b2b2-aa2e-4594-9443-7478733551de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:04.411947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1342/2694] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:04.422092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3a39e-a538-4d92-941d-0554cfe8d1e1 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:04.423495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-94b3a39e-a538-4d92-941d-0554cfe8d1e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:04.479507 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3a39e-a538-4d92-941d-0554cfe8d1e1 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:04.483021 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7134cfc-09e1-424f-a61d-11e721721629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:04.483619 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7134cfc-09e1-424f-a61d-11e721721629 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:04.483900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7134cfc-09e1-424f-a61d-11e721721629 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:04.484177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7134cfc-09e1-424f-a61d-11e721721629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:04.493606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7134cfc-09e1-424f-a61d-11e721721629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:04.495025 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7134cfc-09e1-424f-a61d-11e721721629 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:04.504799 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1353/2695] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:45:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:45:04.931969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30b2b2-aa2e-4594-9443-7478733551de None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:04.933596 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-cb30b2b2-aa2e-4594-9443-7478733551de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:04.946359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30b2b2-aa2e-4594-9443-7478733551de None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:04.951275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd32ea1-e464-45e2-af4b-416073528330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:04.951909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd32ea1-e464-45e2-af4b-416073528330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:04.952192 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd32ea1-e464-45e2-af4b-416073528330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:04.952710 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd32ea1-e464-45e2-af4b-416073528330 admin admin] PATH_INFO: `/v3/projects/2bab654cebb846d2b41b6da2183cd33b` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:04.953999 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd32ea1-e464-45e2-af4b-416073528330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bab654cebb846d2b41b6da2183cd33b)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:04.958181 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd32ea1-e464-45e2-af4b-416073528330 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:04.973772 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:05.090093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4dd32ea1-e464-45e2-af4b-416073528330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bab654cebb846d2b41b6da2183cd33b', 'request_id': 'req-4dd32ea1-e464-45e2-af4b-416073528330'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:05.095522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4dd32ea1-e464-45e2-af4b-416073528330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bab654cebb846d2b41b6da2183cd33b', 'request_id': 'req-4dd32ea1-e464-45e2-af4b-416073528330'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:05.102187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4dd32ea1-e464-45e2-af4b-416073528330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bab654cebb846d2b41b6da2183cd33b', 'request_id': 'req-4dd32ea1-e464-45e2-af4b-416073528330'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:05.191960 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4dd32ea1-e464-45e2-af4b-416073528330 admin admin] The token cache is being invalidate because project 2bab654cebb846d2b41b6da2183cd33b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:45:05.192246 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4dd32ea1-e464-45e2-af4b-416073528330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:05.192883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4dd32ea1-e464-45e2-af4b-416073528330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:05.194415 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1343/2696] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:45:04 2022] DELETE /identity/v3/projects/2bab654cebb846d2b41b6da2183cd33b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:05.198551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7134cfc-09e1-424f-a61d-11e721721629 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:05.208981 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7134cfc-09e1-424f-a61d-11e721721629 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:05.270330 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbe5a6-c84e-4869-9954-732455af8b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:05.271043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbe5a6-c84e-4869-9954-732455af8b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:05.271326 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbe5a6-c84e-4869-9954-732455af8b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:05.271627 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbe5a6-c84e-4869-9954-732455af8b58 admin admin] PATH_INFO: `/v3/users/7e65e3808cda48b4a288db83233c17ef` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:05.273373 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdbe5a6-c84e-4869-9954-732455af8b58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e65e3808cda48b4a288db83233c17ef)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:05.275301 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdbe5a6-c84e-4869-9954-732455af8b58 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:05.485271 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-5fdbe5a6-c84e-4869-9954-732455af8b58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e65e3808cda48b4a288db83233c17ef'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:05.575173 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-5fdbe5a6-c84e-4869-9954-732455af8b58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e65e3808cda48b4a288db83233c17ef'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:05.575738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-5fdbe5a6-c84e-4869-9954-732455af8b58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e65e3808cda48b4a288db83233c17ef'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:05.576366 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-5fdbe5a6-c84e-4869-9954-732455af8b58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e65e3808cda48b4a288db83233c17ef'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:05.584449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-5fdbe5a6-c84e-4869-9954-732455af8b58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e65e3808cda48b4a288db83233c17ef'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:05.597533 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:05.602661 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1354/2697] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:45:05 2022] DELETE /identity/v3/users/7e65e3808cda48b4a288db83233c17ef => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:06.033080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd32ea1-e464-45e2-af4b-416073528330 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:06.034592 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4dd32ea1-e464-45e2-af4b-416073528330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:06.049090 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd32ea1-e464-45e2-af4b-416073528330 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:06.107776 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a2f4b-17c1-4f7b-9f5c-47c0b60a6b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:06.108324 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a2f4b-17c1-4f7b-9f5c-47c0b60a6b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:06.108605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a2f4b-17c1-4f7b-9f5c-47c0b60a6b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:06.108880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a2f4b-17c1-4f7b-9f5c-47c0b60a6b2e admin admin] PATH_INFO: `/v3/projects/5f2666c268ac46bfb6046ab379ee7367` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:06.110088 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a2f4b-17c1-4f7b-9f5c-47c0b60a6b2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f2666c268ac46bfb6046ab379ee7367)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:06.112518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a2f4b-17c1-4f7b-9f5c-47c0b60a6b2e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:06.229052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-499a2f4b-17c1-4f7b-9f5c-47c0b60a6b2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f2666c268ac46bfb6046ab379ee7367', 'request_id': 'req-499a2f4b-17c1-4f7b-9f5c-47c0b60a6b2e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:06.234726 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-499a2f4b-17c1-4f7b-9f5c-47c0b60a6b2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f2666c268ac46bfb6046ab379ee7367', 'request_id': 'req-499a2f4b-17c1-4f7b-9f5c-47c0b60a6b2e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:06.241935 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-499a2f4b-17c1-4f7b-9f5c-47c0b60a6b2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f2666c268ac46bfb6046ab379ee7367', 'request_id': 'req-499a2f4b-17c1-4f7b-9f5c-47c0b60a6b2e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:06.306716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-499a2f4b-17c1-4f7b-9f5c-47c0b60a6b2e admin admin] The token cache is being invalidate because project 5f2666c268ac46bfb6046ab379ee7367 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:45:06.307011 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-499a2f4b-17c1-4f7b-9f5c-47c0b60a6b2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:06.307650 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-499a2f4b-17c1-4f7b-9f5c-47c0b60a6b2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:06.309177 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1344/2698] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:45:06 2022] DELETE /identity/v3/projects/5f2666c268ac46bfb6046ab379ee7367 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:06.313221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbe5a6-c84e-4869-9954-732455af8b58 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:06.314673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5fdbe5a6-c84e-4869-9954-732455af8b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:06.327300 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbe5a6-c84e-4869-9954-732455af8b58 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:06.384861 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9c456-61a4-4313-b9dc-20d71bc4c323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:06.385579 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9c456-61a4-4313-b9dc-20d71bc4c323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:06.385797 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9c456-61a4-4313-b9dc-20d71bc4c323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:06.386071 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9c456-61a4-4313-b9dc-20d71bc4c323 admin admin] PATH_INFO: `/v3/users/ecce0ec6e316455f85ab72753c023813` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:06.387196 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9c456-61a4-4313-b9dc-20d71bc4c323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecce0ec6e316455f85ab72753c023813)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:06.389167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9c456-61a4-4313-b9dc-20d71bc4c323 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:06.571906 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-14f9c456-61a4-4313-b9dc-20d71bc4c323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecce0ec6e316455f85ab72753c023813'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:06.626364 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-14f9c456-61a4-4313-b9dc-20d71bc4c323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecce0ec6e316455f85ab72753c023813'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:06.627003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-14f9c456-61a4-4313-b9dc-20d71bc4c323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecce0ec6e316455f85ab72753c023813'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:06.627597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-14f9c456-61a4-4313-b9dc-20d71bc4c323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecce0ec6e316455f85ab72753c023813'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:06.634875 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-14f9c456-61a4-4313-b9dc-20d71bc4c323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecce0ec6e316455f85ab72753c023813'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:06.654697 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1355/2699] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:45:06 2022] DELETE /identity/v3/users/ecce0ec6e316455f85ab72753c023813 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:07.074441 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a2f4b-17c1-4f7b-9f5c-47c0b60a6b2e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:07.076051 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-499a2f4b-17c1-4f7b-9f5c-47c0b60a6b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:07.091228 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a2f4b-17c1-4f7b-9f5c-47c0b60a6b2e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:07.124941 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:07.149614 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10328e1b-afbc-45b0-9c70-09fd4dd43578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:07.150241 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10328e1b-afbc-45b0-9c70-09fd4dd43578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:07.150521 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10328e1b-afbc-45b0-9c70-09fd4dd43578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:07.150798 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10328e1b-afbc-45b0-9c70-09fd4dd43578 admin admin] PATH_INFO: `/v3/projects/7a436a13d2724a15bdd068af53880ae4` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:07.152000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10328e1b-afbc-45b0-9c70-09fd4dd43578 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a436a13d2724a15bdd068af53880ae4)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:07.153949 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10328e1b-afbc-45b0-9c70-09fd4dd43578 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:07.312762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-10328e1b-afbc-45b0-9c70-09fd4dd43578 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a436a13d2724a15bdd068af53880ae4', 'request_id': 'req-10328e1b-afbc-45b0-9c70-09fd4dd43578'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:07.318815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-10328e1b-afbc-45b0-9c70-09fd4dd43578 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a436a13d2724a15bdd068af53880ae4', 'request_id': 'req-10328e1b-afbc-45b0-9c70-09fd4dd43578'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:07.325223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-10328e1b-afbc-45b0-9c70-09fd4dd43578 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a436a13d2724a15bdd068af53880ae4', 'request_id': 'req-10328e1b-afbc-45b0-9c70-09fd4dd43578'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:07.408684 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-10328e1b-afbc-45b0-9c70-09fd4dd43578 admin admin] The token cache is being invalidate because project 7a436a13d2724a15bdd068af53880ae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:45:07.408973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-10328e1b-afbc-45b0-9c70-09fd4dd43578 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:07.409655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-10328e1b-afbc-45b0-9c70-09fd4dd43578 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:07.411126 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1345/2700] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:45:07 2022] DELETE /identity/v3/projects/7a436a13d2724a15bdd068af53880ae4 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:07.417287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21ec68-ea52-428b-bcd2-056c22588f68 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:07.417782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21ec68-ea52-428b-bcd2-056c22588f68 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:07.417895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21ec68-ea52-428b-bcd2-056c22588f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:07.438256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-3b21ec68-ea52-428b-bcd2-056c22588f68 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:07.439453 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-3b21ec68-ea52-428b-bcd2-056c22588f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:07.500072 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1356/2701] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:07.511983 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8305a-433c-4e4b-bd4b-1d52b1992f28 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:07.512405 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8305a-433c-4e4b-bd4b-1d52b1992f28 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:07.512809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8305a-433c-4e4b-bd4b-1d52b1992f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:07.533295 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-54c8305a-433c-4e4b-bd4b-1d52b1992f28 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:07.534516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-54c8305a-433c-4e4b-bd4b-1d52b1992f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:07.542609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1346/2702] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:07.546966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21ec68-ea52-428b-bcd2-056c22588f68 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:07.557403 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21ec68-ea52-428b-bcd2-056c22588f68 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:07.562109 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7edda-d8fe-42d1-9359-20f9ddc1e368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:07.562833 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7edda-d8fe-42d1-9359-20f9ddc1e368 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:07.563058 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7edda-d8fe-42d1-9359-20f9ddc1e368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:07.563328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7edda-d8fe-42d1-9359-20f9ddc1e368 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:07.563966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7edda-d8fe-42d1-9359-20f9ddc1e368 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:45:07.564340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7edda-d8fe-42d1-9359-20f9ddc1e368 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:07.566651 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7edda-d8fe-42d1-9359-20f9ddc1e368 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:07.578412 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1357/2703] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:45:07.582636 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8305a-433c-4e4b-bd4b-1d52b1992f28 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:07.592154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8305a-433c-4e4b-bd4b-1d52b1992f28 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:07.597051 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa10987-87a4-49a4-b052-8fa3891e0976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:07.597880 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa10987-87a4-49a4-b052-8fa3891e0976 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:07.598164 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa10987-87a4-49a4-b052-8fa3891e0976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:07.598440 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa10987-87a4-49a4-b052-8fa3891e0976 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:07.599085 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa10987-87a4-49a4-b052-8fa3891e0976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:07.601148 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa10987-87a4-49a4-b052-8fa3891e0976 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:07.690952 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1347/2704] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:45:07 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:45:07.695316 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7edda-d8fe-42d1-9359-20f9ddc1e368 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:07.701747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:07.705495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7edda-d8fe-42d1-9359-20f9ddc1e368 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:07.761589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a89815-9127-40b0-a025-db59899d0641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:07.762200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a89815-9127-40b0-a025-db59899d0641 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:07.762483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a89815-9127-40b0-a025-db59899d0641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:07.762756 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a89815-9127-40b0-a025-db59899d0641 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:07.763374 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a89815-9127-40b0-a025-db59899d0641 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:07.765327 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a89815-9127-40b0-a025-db59899d0641 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:07.884066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1358/2705] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:45:07 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:45:07.888036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa10987-87a4-49a4-b052-8fa3891e0976 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:07.898506 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa10987-87a4-49a4-b052-8fa3891e0976 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:07.903260 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d5690-b513-4547-bbf4-7a75725c5a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:07.903853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d5690-b513-4547-bbf4-7a75725c5a86 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:07.904135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d5690-b513-4547-bbf4-7a75725c5a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:07.904410 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d5690-b513-4547-bbf4-7a75725c5a86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:07.905049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d5690-b513-4547-bbf4-7a75725c5a86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:07.906660 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d5690-b513-4547-bbf4-7a75725c5a86 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:07.920955 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1348/2706] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:07.926194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a89815-9127-40b0-a025-db59899d0641 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:07.936644 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a89815-9127-40b0-a025-db59899d0641 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:07.941375 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45740be-af6f-47fa-93be-6d6278bbfd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:07.942005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45740be-af6f-47fa-93be-6d6278bbfd75 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:07.942286 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45740be-af6f-47fa-93be-6d6278bbfd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:07.942560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45740be-af6f-47fa-93be-6d6278bbfd75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:07.943197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45740be-af6f-47fa-93be-6d6278bbfd75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:07.944788 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45740be-af6f-47fa-93be-6d6278bbfd75 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:07.955332 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1359/2707] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:07.959381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d5690-b513-4547-bbf4-7a75725c5a86 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:07.969135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d5690-b513-4547-bbf4-7a75725c5a86 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:07.974132 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b341077-e944-4223-a8d7-796dc3ef6c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:07.974765 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b341077-e944-4223-a8d7-796dc3ef6c6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:07.975037 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b341077-e944-4223-a8d7-796dc3ef6c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:07.975306 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b341077-e944-4223-a8d7-796dc3ef6c6b admin admin] PATH_INFO: `/v3/projects/1af358e393c5476d98dd1a40188bc85f/users/5669d6cca1694f1d9c97d9117053244e/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:07.991825 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b341077-e944-4223-a8d7-796dc3ef6c6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1af358e393c5476d98dd1a40188bc85f, user_id=5669d6cca1694f1d9c97d9117053244e, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:07.994543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b341077-e944-4223-a8d7-796dc3ef6c6b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:08.078340 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1349/2708] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:45:07 2022] PUT /identity/v3/projects/1af358e393c5476d98dd1a40188bc85f/users/5669d6cca1694f1d9c97d9117053244e/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:08.090998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90521f1a-583d-4f71-ae08-7d6da80542d4 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:08.091281 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90521f1a-583d-4f71-ae08-7d6da80542d4 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:08.091564 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90521f1a-583d-4f71-ae08-7d6da80542d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:08.120450 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-90521f1a-583d-4f71-ae08-7d6da80542d4 None None] MFA Rules not processed for user `5669d6cca1694f1d9c97d9117053244e`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:08.121728 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-90521f1a-583d-4f71-ae08-7d6da80542d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:08.187531 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1360/2709] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:08.192037 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b341077-e944-4223-a8d7-796dc3ef6c6b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:08.254110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b341077-e944-4223-a8d7-796dc3ef6c6b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:08.258806 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c3f4b-f4fa-4f4f-93b1-495f4410f4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:08.259706 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c3f4b-f4fa-4f4f-93b1-495f4410f4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:08.260696 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c3f4b-f4fa-4f4f-93b1-495f4410f4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:08.260987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c3f4b-f4fa-4f4f-93b1-495f4410f4a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:08.261705 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c3f4b-f4fa-4f4f-93b1-495f4410f4a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:08.263794 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c3f4b-f4fa-4f4f-93b1-495f4410f4a2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:08.269821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:08.341125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1350/2710] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:45:08 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:45:08.342151 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90521f1a-583d-4f71-ae08-7d6da80542d4 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:08.353606 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90521f1a-583d-4f71-ae08-7d6da80542d4 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:08.415741 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a141d0-8ab2-4788-933e-96a5dbd19d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:08.416335 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a141d0-8ab2-4788-933e-96a5dbd19d1b admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:08.416624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a141d0-8ab2-4788-933e-96a5dbd19d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:08.416898 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a141d0-8ab2-4788-933e-96a5dbd19d1b admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:08.417574 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a141d0-8ab2-4788-933e-96a5dbd19d1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:08.419568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a141d0-8ab2-4788-933e-96a5dbd19d1b admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:08.540720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1361/2711] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:45:08 2022] POST /identity/v3/users => generated 298 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:45:08.545053 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c3f4b-f4fa-4f4f-93b1-495f4410f4a2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:08.555198 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c3f4b-f4fa-4f4f-93b1-495f4410f4a2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:08.559719 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908f388-343f-4bcc-8535-f0b32a4e3d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:08.560310 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908f388-343f-4bcc-8535-f0b32a4e3d17 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:08.560608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908f388-343f-4bcc-8535-f0b32a4e3d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:08.560873 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908f388-343f-4bcc-8535-f0b32a4e3d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:08.561553 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908f388-343f-4bcc-8535-f0b32a4e3d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:08.563141 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908f388-343f-4bcc-8535-f0b32a4e3d17 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:08.573277 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1351/2712] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:08.577320 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a141d0-8ab2-4788-933e-96a5dbd19d1b admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:08.583335 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:08.588615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a141d0-8ab2-4788-933e-96a5dbd19d1b admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:08.594792 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f97b9-8713-49ba-b798-bbc473dcfca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:08.594792 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f97b9-8713-49ba-b798-bbc473dcfca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:08.594792 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f97b9-8713-49ba-b798-bbc473dcfca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:08.594792 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f97b9-8713-49ba-b798-bbc473dcfca3 admin admin] PATH_INFO: `/v3/projects/046daf25c48d4aafaed6200eacf4754e/users/3d7fe26aa1ed44acb020b9f99c2cddac/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:08.609028 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f97b9-8713-49ba-b798-bbc473dcfca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=046daf25c48d4aafaed6200eacf4754e, user_id=3d7fe26aa1ed44acb020b9f99c2cddac, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:08.611700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f97b9-8713-49ba-b798-bbc473dcfca3 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:08.696254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1362/2713] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:45:08 2022] PUT /identity/v3/projects/046daf25c48d4aafaed6200eacf4754e/users/3d7fe26aa1ed44acb020b9f99c2cddac/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:08.702048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908f388-343f-4bcc-8535-f0b32a4e3d17 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:08.712995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908f388-343f-4bcc-8535-f0b32a4e3d17 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:08.772730 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda52d9f-8b12-4d47-b2b0-40e3304690cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:08.773373 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda52d9f-8b12-4d47-b2b0-40e3304690cb admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:08.773775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda52d9f-8b12-4d47-b2b0-40e3304690cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:08.774067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda52d9f-8b12-4d47-b2b0-40e3304690cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:08.774725 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda52d9f-8b12-4d47-b2b0-40e3304690cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:08.776321 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda52d9f-8b12-4d47-b2b0-40e3304690cb admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:08.786320 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1352/2714] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:08.790809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f97b9-8713-49ba-b798-bbc473dcfca3 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:08.803087 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f97b9-8713-49ba-b798-bbc473dcfca3 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:08.807981 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82cbf7-e57d-4d16-bdc8-d6e8ad828bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:08.808747 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82cbf7-e57d-4d16-bdc8-d6e8ad828bed admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:08.809053 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82cbf7-e57d-4d16-bdc8-d6e8ad828bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:08.809362 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82cbf7-e57d-4d16-bdc8-d6e8ad828bed admin admin] PATH_INFO: `/v3/projects/046daf25c48d4aafaed6200eacf4754e/users/3d7fe26aa1ed44acb020b9f99c2cddac/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:08.811539 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82cbf7-e57d-4d16-bdc8-d6e8ad828bed admin admin] RBAC: Authorizing `identity:create_grant(project_id=046daf25c48d4aafaed6200eacf4754e, user_id=3d7fe26aa1ed44acb020b9f99c2cddac, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:08.814535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82cbf7-e57d-4d16-bdc8-d6e8ad828bed admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:08.831240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1363/2715] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:45:08 2022] PUT /identity/v3/projects/046daf25c48d4aafaed6200eacf4754e/users/3d7fe26aa1ed44acb020b9f99c2cddac/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:08.837235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f624257-66b6-4642-b184-1136decebfd7 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:08.837523 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f624257-66b6-4642-b184-1136decebfd7 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:08.837799 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f624257-66b6-4642-b184-1136decebfd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:08.865365 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-6f624257-66b6-4642-b184-1136decebfd7 None None] MFA Rules not processed for user `3d7fe26aa1ed44acb020b9f99c2cddac`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:08.866498 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6f624257-66b6-4642-b184-1136decebfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:08.933843 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1353/2716] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:08 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:08.946550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82cbf7-e57d-4d16-bdc8-d6e8ad828bed admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:08.947869 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-3d82cbf7-e57d-4d16-bdc8-d6e8ad828bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:09.005163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82cbf7-e57d-4d16-bdc8-d6e8ad828bed admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:09.008812 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5080fd-bc7a-4d71-b831-ef21b35347a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:09.009487 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5080fd-bc7a-4d71-b831-ef21b35347a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:09.009764 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5080fd-bc7a-4d71-b831-ef21b35347a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:09.010031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5080fd-bc7a-4d71-b831-ef21b35347a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:09.019305 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5080fd-bc7a-4d71-b831-ef21b35347a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:09.020758 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5080fd-bc7a-4d71-b831-ef21b35347a1 service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:09.031118 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1364/2717] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:45:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:45:09.184682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f624257-66b6-4642-b184-1136decebfd7 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:09.194344 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f624257-66b6-4642-b184-1136decebfd7 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:09.197975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218f17b-d0d4-4729-9d1e-92a24a29856e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:09.198564 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218f17b-d0d4-4729-9d1e-92a24a29856e service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:09.198840 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218f17b-d0d4-4729-9d1e-92a24a29856e service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:09.199107 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218f17b-d0d4-4729-9d1e-92a24a29856e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:09.253685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:09.255845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218f17b-d0d4-4729-9d1e-92a24a29856e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:09.257315 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218f17b-d0d4-4729-9d1e-92a24a29856e service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:09.266103 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1354/2718] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:45:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:45:09.779405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5080fd-bc7a-4d71-b831-ef21b35347a1 service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:09.848201 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5080fd-bc7a-4d71-b831-ef21b35347a1 service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:09.852719 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c22aa2-4607-463f-951f-a87f85f408a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:09.853309 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c22aa2-4607-463f-951f-a87f85f408a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:09.853611 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c22aa2-4607-463f-951f-a87f85f408a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:09.853881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c22aa2-4607-463f-951f-a87f85f408a2 admin admin] PATH_INFO: `/v3/users/5669d6cca1694f1d9c97d9117053244e` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:09.855138 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c22aa2-4607-463f-951f-a87f85f408a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5669d6cca1694f1d9c97d9117053244e)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:09.857222 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c22aa2-4607-463f-951f-a87f85f408a2 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:10.092699 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-d7c22aa2-4607-463f-951f-a87f85f408a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5669d6cca1694f1d9c97d9117053244e'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:10.225649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-d7c22aa2-4607-463f-951f-a87f85f408a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5669d6cca1694f1d9c97d9117053244e'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:10.226271 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-d7c22aa2-4607-463f-951f-a87f85f408a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5669d6cca1694f1d9c97d9117053244e'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:10.226928 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-d7c22aa2-4607-463f-951f-a87f85f408a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5669d6cca1694f1d9c97d9117053244e'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:10.234441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-d7c22aa2-4607-463f-951f-a87f85f408a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5669d6cca1694f1d9c97d9117053244e'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:10.239793 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:10.262015 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1365/2719] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:45:09 2022] DELETE /identity/v3/users/5669d6cca1694f1d9c97d9117053244e => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:10.266893 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea73a0e-b03a-4f91-ac2d-c2bac351a71c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:10.267404 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea73a0e-b03a-4f91-ac2d-c2bac351a71c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:10.267404 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea73a0e-b03a-4f91-ac2d-c2bac351a71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:10.287582 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-aea73a0e-b03a-4f91-ac2d-c2bac351a71c None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:10.288783 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-aea73a0e-b03a-4f91-ac2d-c2bac351a71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:10.348315 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1355/2720] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:10.358318 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c22aa2-4607-463f-951f-a87f85f408a2 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:10.359744 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-d7c22aa2-4607-463f-951f-a87f85f408a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:10.417235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c22aa2-4607-463f-951f-a87f85f408a2 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:10.420955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e53ff6-58f1-4622-90d3-620702d8ad4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:10.421572 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e53ff6-58f1-4622-90d3-620702d8ad4b service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:10.421849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e53ff6-58f1-4622-90d3-620702d8ad4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:10.422121 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e53ff6-58f1-4622-90d3-620702d8ad4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:10.431487 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e53ff6-58f1-4622-90d3-620702d8ad4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:10.432901 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e53ff6-58f1-4622-90d3-620702d8ad4b service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:10.442954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1366/2721] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:45:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:45:10.868633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea73a0e-b03a-4f91-ac2d-c2bac351a71c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:10.870381 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-aea73a0e-b03a-4f91-ac2d-c2bac351a71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:10.888553 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea73a0e-b03a-4f91-ac2d-c2bac351a71c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:10.894332 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06557b-5f70-4aaa-8249-0b0c641ae83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:10.895055 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06557b-5f70-4aaa-8249-0b0c641ae83e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:10.895359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06557b-5f70-4aaa-8249-0b0c641ae83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:10.895665 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06557b-5f70-4aaa-8249-0b0c641ae83e admin admin] PATH_INFO: `/v3/projects/1af358e393c5476d98dd1a40188bc85f` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:10.896719 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06557b-5f70-4aaa-8249-0b0c641ae83e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1af358e393c5476d98dd1a40188bc85f)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:10.904084 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06557b-5f70-4aaa-8249-0b0c641ae83e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:11.067551 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-2b06557b-5f70-4aaa-8249-0b0c641ae83e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1af358e393c5476d98dd1a40188bc85f', 'request_id': 'req-2b06557b-5f70-4aaa-8249-0b0c641ae83e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:11.073314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-2b06557b-5f70-4aaa-8249-0b0c641ae83e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1af358e393c5476d98dd1a40188bc85f', 'request_id': 'req-2b06557b-5f70-4aaa-8249-0b0c641ae83e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:11.079716 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-2b06557b-5f70-4aaa-8249-0b0c641ae83e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1af358e393c5476d98dd1a40188bc85f', 'request_id': 'req-2b06557b-5f70-4aaa-8249-0b0c641ae83e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:11.192556 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-2b06557b-5f70-4aaa-8249-0b0c641ae83e admin admin] The token cache is being invalidate because project 1af358e393c5476d98dd1a40188bc85f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:45:11.192868 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-2b06557b-5f70-4aaa-8249-0b0c641ae83e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:11.193509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-2b06557b-5f70-4aaa-8249-0b0c641ae83e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:11.194985 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1356/2722] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:45:10 2022] DELETE /identity/v3/projects/1af358e393c5476d98dd1a40188bc85f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:11.199157 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e53ff6-58f1-4622-90d3-620702d8ad4b service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:11.208885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e53ff6-58f1-4622-90d3-620702d8ad4b service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:11.291065 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261be85c-48c4-4da9-b59d-c64fc590e75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:11.291702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261be85c-48c4-4da9-b59d-c64fc590e75d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:11.292045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261be85c-48c4-4da9-b59d-c64fc590e75d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:11.292320 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261be85c-48c4-4da9-b59d-c64fc590e75d admin admin] PATH_INFO: `/v3/users/3d7fe26aa1ed44acb020b9f99c2cddac` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:11.293475 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261be85c-48c4-4da9-b59d-c64fc590e75d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d7fe26aa1ed44acb020b9f99c2cddac)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:11.295453 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261be85c-48c4-4da9-b59d-c64fc590e75d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:11.504727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-261be85c-48c4-4da9-b59d-c64fc590e75d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d7fe26aa1ed44acb020b9f99c2cddac'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:11.574826 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-261be85c-48c4-4da9-b59d-c64fc590e75d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d7fe26aa1ed44acb020b9f99c2cddac'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:11.575437 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-261be85c-48c4-4da9-b59d-c64fc590e75d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d7fe26aa1ed44acb020b9f99c2cddac'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:11.576046 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-261be85c-48c4-4da9-b59d-c64fc590e75d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d7fe26aa1ed44acb020b9f99c2cddac'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:11.583541 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-261be85c-48c4-4da9-b59d-c64fc590e75d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d7fe26aa1ed44acb020b9f99c2cddac'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:11.601145 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1367/2723] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:45:11 2022] DELETE /identity/v3/users/3d7fe26aa1ed44acb020b9f99c2cddac => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:11.993979 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06557b-5f70-4aaa-8249-0b0c641ae83e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:11.995437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2b06557b-5f70-4aaa-8249-0b0c641ae83e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:12.008708 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06557b-5f70-4aaa-8249-0b0c641ae83e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:12.067728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304a48c-f1b5-4fe0-9ab2-a50622349d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:12.068328 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304a48c-f1b5-4fe0-9ab2-a50622349d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:12.068629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304a48c-f1b5-4fe0-9ab2-a50622349d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:12.068888 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304a48c-f1b5-4fe0-9ab2-a50622349d9a admin admin] PATH_INFO: `/v3/projects/046daf25c48d4aafaed6200eacf4754e` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:12.070042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304a48c-f1b5-4fe0-9ab2-a50622349d9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=046daf25c48d4aafaed6200eacf4754e)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:12.071940 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304a48c-f1b5-4fe0-9ab2-a50622349d9a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:12.094656 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:12.192522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e304a48c-f1b5-4fe0-9ab2-a50622349d9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '046daf25c48d4aafaed6200eacf4754e', 'request_id': 'req-e304a48c-f1b5-4fe0-9ab2-a50622349d9a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:12.198619 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e304a48c-f1b5-4fe0-9ab2-a50622349d9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '046daf25c48d4aafaed6200eacf4754e', 'request_id': 'req-e304a48c-f1b5-4fe0-9ab2-a50622349d9a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:12.204735 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e304a48c-f1b5-4fe0-9ab2-a50622349d9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '046daf25c48d4aafaed6200eacf4754e', 'request_id': 'req-e304a48c-f1b5-4fe0-9ab2-a50622349d9a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:12.294940 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e304a48c-f1b5-4fe0-9ab2-a50622349d9a admin admin] The token cache is being invalidate because project 046daf25c48d4aafaed6200eacf4754e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:45:12.295237 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e304a48c-f1b5-4fe0-9ab2-a50622349d9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:12.295878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e304a48c-f1b5-4fe0-9ab2-a50622349d9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:12.297411 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1357/2724] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:45:11 2022] DELETE /identity/v3/projects/046daf25c48d4aafaed6200eacf4754e => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:12.302135 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a64a0-29c2-42f7-b8a7-e584beb47d46 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:12.302618 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a64a0-29c2-42f7-b8a7-e584beb47d46 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:12.302802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a64a0-29c2-42f7-b8a7-e584beb47d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:12.322927 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-dc1a64a0-29c2-42f7-b8a7-e584beb47d46 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:12.324132 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-dc1a64a0-29c2-42f7-b8a7-e584beb47d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:12.328452 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:12.398713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1368/2725] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:45:12.467349 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ff3f7-8f7f-493d-a042-e0dc3c572146 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:12.467349 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ff3f7-8f7f-493d-a042-e0dc3c572146 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:12.467349 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ff3f7-8f7f-493d-a042-e0dc3c572146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:12.489828 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-c77ff3f7-8f7f-493d-a042-e0dc3c572146 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:12.493584 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-c77ff3f7-8f7f-493d-a042-e0dc3c572146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:12.499220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1358/2726] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:12.504400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a64a0-29c2-42f7-b8a7-e584beb47d46 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:12.514852 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a64a0-29c2-42f7-b8a7-e584beb47d46 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:12.519512 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0c1bc-6e83-4f00-b938-d60efaadc248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:12.520112 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0c1bc-6e83-4f00-b938-d60efaadc248 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:12.522427 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0c1bc-6e83-4f00-b938-d60efaadc248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:12.522427 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0c1bc-6e83-4f00-b938-d60efaadc248 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:12.522427 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0c1bc-6e83-4f00-b938-d60efaadc248 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:45:12.522427 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0c1bc-6e83-4f00-b938-d60efaadc248 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:12.523499 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0c1bc-6e83-4f00-b938-d60efaadc248 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:12.535099 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1369/2727] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:45:12.539447 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ff3f7-8f7f-493d-a042-e0dc3c572146 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:12.548558 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ff3f7-8f7f-493d-a042-e0dc3c572146 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:12.553855 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b9a3a-defe-4e7c-9bbd-5e9c240d2c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:12.554464 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b9a3a-defe-4e7c-9bbd-5e9c240d2c40 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:12.554731 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b9a3a-defe-4e7c-9bbd-5e9c240d2c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:12.555005 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b9a3a-defe-4e7c-9bbd-5e9c240d2c40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:12.555646 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b9a3a-defe-4e7c-9bbd-5e9c240d2c40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:12.557577 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b9a3a-defe-4e7c-9bbd-5e9c240d2c40 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:12.662519 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1359/2728] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:45:12 2022] POST /identity/v3/projects => generated 369 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:45:12.669287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0c1bc-6e83-4f00-b938-d60efaadc248 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:12.680628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0c1bc-6e83-4f00-b938-d60efaadc248 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:12.741121 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf6d29-f733-4eb3-a6de-36c4dd515da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:12.741661 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf6d29-f733-4eb3-a6de-36c4dd515da0 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:12.741943 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf6d29-f733-4eb3-a6de-36c4dd515da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:12.742220 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf6d29-f733-4eb3-a6de-36c4dd515da0 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:12.742853 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf6d29-f733-4eb3-a6de-36c4dd515da0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:12.744836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf6d29-f733-4eb3-a6de-36c4dd515da0 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:12.844829 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1370/2729] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:45:12 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:45:12.851067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b9a3a-defe-4e7c-9bbd-5e9c240d2c40 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:12.861323 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b9a3a-defe-4e7c-9bbd-5e9c240d2c40 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:12.866656 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06f9ab-ae2b-4ac0-8ecb-3241ec30ab26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:12.871120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06f9ab-ae2b-4ac0-8ecb-3241ec30ab26 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:12.871120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06f9ab-ae2b-4ac0-8ecb-3241ec30ab26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:12.871120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06f9ab-ae2b-4ac0-8ecb-3241ec30ab26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:12.871120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06f9ab-ae2b-4ac0-8ecb-3241ec30ab26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:12.871120 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06f9ab-ae2b-4ac0-8ecb-3241ec30ab26 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:12.881420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1360/2730] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:12.885676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf6d29-f733-4eb3-a6de-36c4dd515da0 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:12.895856 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf6d29-f733-4eb3-a6de-36c4dd515da0 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:12.900798 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb9e27-f343-40bc-98ef-a591c5711d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:12.901453 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb9e27-f343-40bc-98ef-a591c5711d5c admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:12.901750 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb9e27-f343-40bc-98ef-a591c5711d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:12.902017 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb9e27-f343-40bc-98ef-a591c5711d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:12.902695 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb9e27-f343-40bc-98ef-a591c5711d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:12.904299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb9e27-f343-40bc-98ef-a591c5711d5c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:12.916457 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1371/2731] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:12.920595 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06f9ab-ae2b-4ac0-8ecb-3241ec30ab26 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:12.931188 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06f9ab-ae2b-4ac0-8ecb-3241ec30ab26 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:12.937481 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ba11c-baa0-4ae4-89ee-947672c0a47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:12.938125 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ba11c-baa0-4ae4-89ee-947672c0a47c admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:12.938403 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ba11c-baa0-4ae4-89ee-947672c0a47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:12.938677 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ba11c-baa0-4ae4-89ee-947672c0a47c admin admin] PATH_INFO: `/v3/projects/bd4a147e391448a29f738cb448e9f7bd/users/9f3daf4b24194b81bbf0248ba03b3d20/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:12.953337 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ba11c-baa0-4ae4-89ee-947672c0a47c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd4a147e391448a29f738cb448e9f7bd, user_id=9f3daf4b24194b81bbf0248ba03b3d20, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:12.955989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ba11c-baa0-4ae4-89ee-947672c0a47c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:13.061422 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1361/2732] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:45:12 2022] PUT /identity/v3/projects/bd4a147e391448a29f738cb448e9f7bd/users/9f3daf4b24194b81bbf0248ba03b3d20/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:13.071481 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962f837-dc1e-43d9-930b-497641e81bff None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:13.071768 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962f837-dc1e-43d9-930b-497641e81bff None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:13.072048 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962f837-dc1e-43d9-930b-497641e81bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:13.101777 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-2962f837-dc1e-43d9-930b-497641e81bff None None] MFA Rules not processed for user `9f3daf4b24194b81bbf0248ba03b3d20`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:13.103026 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-2962f837-dc1e-43d9-930b-497641e81bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:13.166318 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1372/2733] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:13.177763 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ba11c-baa0-4ae4-89ee-947672c0a47c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:13.179465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-2e0ba11c-baa0-4ae4-89ee-947672c0a47c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:13.234576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ba11c-baa0-4ae4-89ee-947672c0a47c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:13.238766 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af2137-9af3-4805-a97a-e1b7dc0233ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:13.239405 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af2137-9af3-4805-a97a-e1b7dc0233ed service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:13.239699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af2137-9af3-4805-a97a-e1b7dc0233ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:13.239967 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af2137-9af3-4805-a97a-e1b7dc0233ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:13.264738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af2137-9af3-4805-a97a-e1b7dc0233ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:13.264738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af2137-9af3-4805-a97a-e1b7dc0233ed service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:13.265082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1362/2734] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:45:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:45:23.163403 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:23.229665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962f837-dc1e-43d9-930b-497641e81bff None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:23.260782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:DELETE=7,keystone:SELECT=68,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:23.292249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962f837-dc1e-43d9-930b-497641e81bff None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:23.302549 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44200952-374d-4450-8d54-66fb00f6948c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:23.302549 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44200952-374d-4450-8d54-66fb00f6948c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:23.302549 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44200952-374d-4450-8d54-66fb00f6948c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:23.302549 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44200952-374d-4450-8d54-66fb00f6948c admin admin] PATH_INFO: `/v3/users/9f3daf4b24194b81bbf0248ba03b3d20` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:23.302978 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44200952-374d-4450-8d54-66fb00f6948c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f3daf4b24194b81bbf0248ba03b3d20)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:23.302978 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44200952-374d-4450-8d54-66fb00f6948c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:23.668873 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-44200952-374d-4450-8d54-66fb00f6948c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f3daf4b24194b81bbf0248ba03b3d20'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:23.792585 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-44200952-374d-4450-8d54-66fb00f6948c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f3daf4b24194b81bbf0248ba03b3d20'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:23.793282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-44200952-374d-4450-8d54-66fb00f6948c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f3daf4b24194b81bbf0248ba03b3d20'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:23.793894 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-44200952-374d-4450-8d54-66fb00f6948c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f3daf4b24194b81bbf0248ba03b3d20'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:23.801498 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-44200952-374d-4450-8d54-66fb00f6948c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f3daf4b24194b81bbf0248ba03b3d20'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:23.819624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1373/2735] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:45:23 2022] DELETE /identity/v3/users/9f3daf4b24194b81bbf0248ba03b3d20 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:23.824159 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a6d7c-4d3d-4428-be28-198fc683ce0b None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:23.824445 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a6d7c-4d3d-4428-be28-198fc683ce0b None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:23.824728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a6d7c-4d3d-4428-be28-198fc683ce0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:23.847845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-da9a6d7c-4d3d-4428-be28-198fc683ce0b None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:23.847845 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-da9a6d7c-4d3d-4428-be28-198fc683ce0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:23.903848 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1363/2736] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:45:23.913645 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44200952-374d-4450-8d54-66fb00f6948c admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:23.915047 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-44200952-374d-4450-8d54-66fb00f6948c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:23.975787 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44200952-374d-4450-8d54-66fb00f6948c admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:23.979355 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c78f521-0867-4946-8571-053978c81b7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:23.980519 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78f521-0867-4946-8571-053978c81b7a service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:23.980798 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78f521-0867-4946-8571-053978c81b7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:23.981083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78f521-0867-4946-8571-053978c81b7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:23.990194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78f521-0867-4946-8571-053978c81b7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:23.991625 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78f521-0867-4946-8571-053978c81b7a service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:24.000967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1374/2737] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:45:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:45:24.451396 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a6d7c-4d3d-4428-be28-198fc683ce0b None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:24.452916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-da9a6d7c-4d3d-4428-be28-198fc683ce0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:24.466187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a6d7c-4d3d-4428-be28-198fc683ce0b None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:24.473147 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde5e2e-5cfc-4d0c-9ece-78c09a35d75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:24.473997 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde5e2e-5cfc-4d0c-9ece-78c09a35d75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:24.474468 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde5e2e-5cfc-4d0c-9ece-78c09a35d75a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:24.474742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde5e2e-5cfc-4d0c-9ece-78c09a35d75a admin admin] PATH_INFO: `/v3/projects/bd4a147e391448a29f738cb448e9f7bd` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:24.476301 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde5e2e-5cfc-4d0c-9ece-78c09a35d75a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd4a147e391448a29f738cb448e9f7bd)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:24.478585 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde5e2e-5cfc-4d0c-9ece-78c09a35d75a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:24.632854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4dde5e2e-5cfc-4d0c-9ece-78c09a35d75a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd4a147e391448a29f738cb448e9f7bd', 'request_id': 'req-4dde5e2e-5cfc-4d0c-9ece-78c09a35d75a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:24.639692 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4dde5e2e-5cfc-4d0c-9ece-78c09a35d75a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd4a147e391448a29f738cb448e9f7bd', 'request_id': 'req-4dde5e2e-5cfc-4d0c-9ece-78c09a35d75a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:24.646024 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4dde5e2e-5cfc-4d0c-9ece-78c09a35d75a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd4a147e391448a29f738cb448e9f7bd', 'request_id': 'req-4dde5e2e-5cfc-4d0c-9ece-78c09a35d75a'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:24.752123 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4dde5e2e-5cfc-4d0c-9ece-78c09a35d75a admin admin] The token cache is being invalidate because project bd4a147e391448a29f738cb448e9f7bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:45:24.752357 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4dde5e2e-5cfc-4d0c-9ece-78c09a35d75a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:24.753245 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-4dde5e2e-5cfc-4d0c-9ece-78c09a35d75a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:24.755130 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1364/2738] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:45:24 2022] DELETE /identity/v3/projects/bd4a147e391448a29f738cb448e9f7bd => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:24.760164 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac2996-e1ca-4934-a2fb-860687213a3a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:24.760588 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac2996-e1ca-4934-a2fb-860687213a3a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:24.760721 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac2996-e1ca-4934-a2fb-860687213a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:24.784292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-05ac2996-e1ca-4934-a2fb-860687213a3a None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:24.785733 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-05ac2996-e1ca-4934-a2fb-860687213a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:24.813869 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:24.855421 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1375/2739] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:24.863113 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446d09e-272c-4c07-85e9-c823033a3110 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:24.863423 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446d09e-272c-4c07-85e9-c823033a3110 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:24.863722 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446d09e-272c-4c07-85e9-c823033a3110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:24.883842 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-3446d09e-272c-4c07-85e9-c823033a3110 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:24.885022 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3446d09e-272c-4c07-85e9-c823033a3110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:24.893615 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1365/2740] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:24.898103 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac2996-e1ca-4934-a2fb-860687213a3a None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:24.907919 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac2996-e1ca-4934-a2fb-860687213a3a None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:24.912536 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d91c9f-1b93-45a7-9083-7f96ceb687ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:24.913170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d91c9f-1b93-45a7-9083-7f96ceb687ae admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:24.913460 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d91c9f-1b93-45a7-9083-7f96ceb687ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:24.913738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d91c9f-1b93-45a7-9083-7f96ceb687ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:24.914386 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d91c9f-1b93-45a7-9083-7f96ceb687ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:45:24.914698 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d91c9f-1b93-45a7-9083-7f96ceb687ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:24.916331 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d91c9f-1b93-45a7-9083-7f96ceb687ae admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:24.928298 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1376/2741] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:45:24.933776 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446d09e-272c-4c07-85e9-c823033a3110 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:24.942124 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446d09e-272c-4c07-85e9-c823033a3110 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:24.946786 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4923ab-91a2-43ed-bbc0-03cf2ad46ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:24.947375 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4923ab-91a2-43ed-bbc0-03cf2ad46ea0 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:24.947650 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4923ab-91a2-43ed-bbc0-03cf2ad46ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:24.947921 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4923ab-91a2-43ed-bbc0-03cf2ad46ea0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:24.949202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4923ab-91a2-43ed-bbc0-03cf2ad46ea0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:24.951167 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4923ab-91a2-43ed-bbc0-03cf2ad46ea0 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:25.074309 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1366/2742] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:45:24 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:45:25.078495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d91c9f-1b93-45a7-9083-7f96ceb687ae admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:25.088662 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d91c9f-1b93-45a7-9083-7f96ceb687ae admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:25.146429 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458f4d1-e0be-40d8-8d3f-5af031545081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:25.147083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458f4d1-e0be-40d8-8d3f-5af031545081 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:25.147614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458f4d1-e0be-40d8-8d3f-5af031545081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:25.147886 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458f4d1-e0be-40d8-8d3f-5af031545081 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:25.148505 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458f4d1-e0be-40d8-8d3f-5af031545081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:25.150565 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458f4d1-e0be-40d8-8d3f-5af031545081 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:25.238364 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1377/2743] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:45:25 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:45:25.242566 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4923ab-91a2-43ed-bbc0-03cf2ad46ea0 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:25.253520 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4923ab-91a2-43ed-bbc0-03cf2ad46ea0 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:25.258866 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2dddd-c1f9-49c6-be8d-7d93a65710a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:25.259486 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2dddd-c1f9-49c6-be8d-7d93a65710a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:25.259766 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2dddd-c1f9-49c6-be8d-7d93a65710a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:25.260042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2dddd-c1f9-49c6-be8d-7d93a65710a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:25.260695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2dddd-c1f9-49c6-be8d-7d93a65710a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:25.262382 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2dddd-c1f9-49c6-be8d-7d93a65710a0 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:25.274060 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1367/2744] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:25.278062 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458f4d1-e0be-40d8-8d3f-5af031545081 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:25.288448 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458f4d1-e0be-40d8-8d3f-5af031545081 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:25.293608 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d170c-9b3a-440a-b797-93013cfc92aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:25.297154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d170c-9b3a-440a-b797-93013cfc92aa admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:25.297154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d170c-9b3a-440a-b797-93013cfc92aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:25.297154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d170c-9b3a-440a-b797-93013cfc92aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:25.297154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d170c-9b3a-440a-b797-93013cfc92aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:25.297154 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d170c-9b3a-440a-b797-93013cfc92aa admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:25.309027 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1378/2745] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:25.312893 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2dddd-c1f9-49c6-be8d-7d93a65710a0 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:25.323180 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2dddd-c1f9-49c6-be8d-7d93a65710a0 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:25.328408 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524b13a-753f-4c81-9f0d-81fadb97430e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:25.328987 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524b13a-753f-4c81-9f0d-81fadb97430e admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:25.329303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524b13a-753f-4c81-9f0d-81fadb97430e admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:25.329899 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524b13a-753f-4c81-9f0d-81fadb97430e admin admin] PATH_INFO: `/v3/projects/fe82ada6f5f34dac92c3df01a052a827/users/7dab6b337b5a4124afece3118ff3d51e/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:25.344762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524b13a-753f-4c81-9f0d-81fadb97430e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe82ada6f5f34dac92c3df01a052a827, user_id=7dab6b337b5a4124afece3118ff3d51e, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:25.347492 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524b13a-753f-4c81-9f0d-81fadb97430e admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:25.404655 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1368/2746] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:45:25 2022] PUT /identity/v3/projects/fe82ada6f5f34dac92c3df01a052a827/users/7dab6b337b5a4124afece3118ff3d51e/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:25.416539 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913c4e9-ff96-459b-adea-7a6fc4eca7ac None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:25.416824 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913c4e9-ff96-459b-adea-7a6fc4eca7ac None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:25.417124 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913c4e9-ff96-459b-adea-7a6fc4eca7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:25.446253 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-1913c4e9-ff96-459b-adea-7a6fc4eca7ac None None] MFA Rules not processed for user `7dab6b337b5a4124afece3118ff3d51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:25.447742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-1913c4e9-ff96-459b-adea-7a6fc4eca7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:25.512841 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1379/2747] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:25.517374 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524b13a-753f-4c81-9f0d-81fadb97430e admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:25.538998 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:25.580267 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524b13a-753f-4c81-9f0d-81fadb97430e admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:25.585398 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98ad6e-633f-4e90-a59f-0644604b9521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:25.586000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98ad6e-633f-4e90-a59f-0644604b9521 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:25.586285 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98ad6e-633f-4e90-a59f-0644604b9521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:25.586561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98ad6e-633f-4e90-a59f-0644604b9521 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:25.587198 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98ad6e-633f-4e90-a59f-0644604b9521 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:25.589158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98ad6e-633f-4e90-a59f-0644604b9521 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:25.678898 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1369/2748] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:45:25 2022] POST /identity/v3/projects => generated 383 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:45:25.683148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913c4e9-ff96-459b-adea-7a6fc4eca7ac None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:25.694625 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913c4e9-ff96-459b-adea-7a6fc4eca7ac None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:25.707816 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:25.752395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d519e27-919a-4f0c-a8ab-9f61de9a7ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:25.753000 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d519e27-919a-4f0c-a8ab-9f61de9a7ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:25.753309 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d519e27-919a-4f0c-a8ab-9f61de9a7ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:25.753589 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d519e27-919a-4f0c-a8ab-9f61de9a7ac4 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:25.754276 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d519e27-919a-4f0c-a8ab-9f61de9a7ac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:25.756194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d519e27-919a-4f0c-a8ab-9f61de9a7ac4 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:25.884668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1380/2749] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:45:25 2022] POST /identity/v3/users => generated 292 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:45:25.889264 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98ad6e-633f-4e90-a59f-0644604b9521 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:25.900050 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98ad6e-633f-4e90-a59f-0644604b9521 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:25.905230 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd180e7-588a-4f9b-9174-147f711a727b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:25.905839 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd180e7-588a-4f9b-9174-147f711a727b admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:25.906251 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd180e7-588a-4f9b-9174-147f711a727b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:25.906452 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd180e7-588a-4f9b-9174-147f711a727b admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:25.907049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd180e7-588a-4f9b-9174-147f711a727b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:25.908634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd180e7-588a-4f9b-9174-147f711a727b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:25.919631 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1370/2750] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:45:25.923757 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d519e27-919a-4f0c-a8ab-9f61de9a7ac4 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:25.934563 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d519e27-919a-4f0c-a8ab-9f61de9a7ac4 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:25.939307 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caea27-6b65-4baa-ad64-0210a71ae219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:25.939840 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caea27-6b65-4baa-ad64-0210a71ae219 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:25.940147 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caea27-6b65-4baa-ad64-0210a71ae219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:25.940381 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caea27-6b65-4baa-ad64-0210a71ae219 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:25.941052 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09caea27-6b65-4baa-ad64-0210a71ae219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:25.942669 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09caea27-6b65-4baa-ad64-0210a71ae219 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:25.953624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1381/2751] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:25.957765 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd180e7-588a-4f9b-9174-147f711a727b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:25.968268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd180e7-588a-4f9b-9174-147f711a727b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:25.972850 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a3d08-5aac-44cf-ac8a-dbf7fe332f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:25.973542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a3d08-5aac-44cf-ac8a-dbf7fe332f10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:25.973823 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a3d08-5aac-44cf-ac8a-dbf7fe332f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:25.977680 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a3d08-5aac-44cf-ac8a-dbf7fe332f10 admin admin] PATH_INFO: `/v3/projects/1a7bac3c4746437abd9041203e42fed8/users/618a9960f0f5485bb9d4cd88651fd767/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:25.989508 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a3d08-5aac-44cf-ac8a-dbf7fe332f10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a7bac3c4746437abd9041203e42fed8, user_id=618a9960f0f5485bb9d4cd88651fd767, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:25.992151 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a3d08-5aac-44cf-ac8a-dbf7fe332f10 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:26.158452 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1371/2752] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:45:25 2022] PUT /identity/v3/projects/1a7bac3c4746437abd9041203e42fed8/users/618a9960f0f5485bb9d4cd88651fd767/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:45:26.169160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64272af0-43bd-4d4e-9049-faf35d543102 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:26.169443 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64272af0-43bd-4d4e-9049-faf35d543102 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:26.169713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64272af0-43bd-4d4e-9049-faf35d543102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:26.198071 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-64272af0-43bd-4d4e-9049-faf35d543102 None None] MFA Rules not processed for user `618a9960f0f5485bb9d4cd88651fd767`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:26.199258 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-64272af0-43bd-4d4e-9049-faf35d543102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:26.267627 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1382/2753] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:26.275401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a3d08-5aac-44cf-ac8a-dbf7fe332f10 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:26.348010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a3d08-5aac-44cf-ac8a-dbf7fe332f10 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:26.352604 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137aba2f-3911-47fb-858d-8f9ee504a6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:26.353254 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137aba2f-3911-47fb-858d-8f9ee504a6fe admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:26.353539 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137aba2f-3911-47fb-858d-8f9ee504a6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:26.353817 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137aba2f-3911-47fb-858d-8f9ee504a6fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:26.354466 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137aba2f-3911-47fb-858d-8f9ee504a6fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:26.357485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137aba2f-3911-47fb-858d-8f9ee504a6fe admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:26.485380 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1372/2754] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:45:26 2022] POST /identity/v3/projects => generated 385 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:45:26.489247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64272af0-43bd-4d4e-9049-faf35d543102 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:26.501684 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64272af0-43bd-4d4e-9049-faf35d543102 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:26.559782 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd1e70-da53-431a-ad2e-59bc0f7b8265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:26.560428 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd1e70-da53-431a-ad2e-59bc0f7b8265 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:26.560726 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd1e70-da53-431a-ad2e-59bc0f7b8265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:26.561000 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd1e70-da53-431a-ad2e-59bc0f7b8265 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:26.561803 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd1e70-da53-431a-ad2e-59bc0f7b8265 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:26.564342 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd1e70-da53-431a-ad2e-59bc0f7b8265 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:26.792983 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1383/2755] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:45:26 2022] POST /identity/v3/users => generated 299 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:45:26.796968 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137aba2f-3911-47fb-858d-8f9ee504a6fe admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:26.807731 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137aba2f-3911-47fb-858d-8f9ee504a6fe admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:26.812568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de934ec-c73b-4577-86d6-2e373a97c16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:26.813214 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de934ec-c73b-4577-86d6-2e373a97c16b admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:26.813562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de934ec-c73b-4577-86d6-2e373a97c16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:26.813836 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de934ec-c73b-4577-86d6-2e373a97c16b admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:26.814486 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de934ec-c73b-4577-86d6-2e373a97c16b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:26.816075 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de934ec-c73b-4577-86d6-2e373a97c16b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:26.827645 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1373/2756] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:26.831841 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd1e70-da53-431a-ad2e-59bc0f7b8265 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:26.842332 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd1e70-da53-431a-ad2e-59bc0f7b8265 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:26.846900 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd32e12-1171-469e-b45f-4966d1faa0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:26.848764 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd32e12-1171-469e-b45f-4966d1faa0b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:26.848764 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd32e12-1171-469e-b45f-4966d1faa0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:26.848764 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd32e12-1171-469e-b45f-4966d1faa0b8 admin admin] PATH_INFO: `/v3/projects/78e61b9be06343df871b827751a94eb8/users/b4caed226c284576921fa2252e1035b7/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:26.860792 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:26.863789 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd32e12-1171-469e-b45f-4966d1faa0b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78e61b9be06343df871b827751a94eb8, user_id=b4caed226c284576921fa2252e1035b7, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:26.866495 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd32e12-1171-469e-b45f-4966d1faa0b8 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:26.991911 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1384/2757] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:45:26 2022] PUT /identity/v3/projects/78e61b9be06343df871b827751a94eb8/users/b4caed226c284576921fa2252e1035b7/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:26.995901 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de934ec-c73b-4577-86d6-2e373a97c16b admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:27.006453 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de934ec-c73b-4577-86d6-2e373a97c16b admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:27.048303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:27.063334 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b7346-bfa9-4b70-a3e9-d905d742b44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:27.063934 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b7346-bfa9-4b70-a3e9-d905d742b44b admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:27.064212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b7346-bfa9-4b70-a3e9-d905d742b44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:27.064510 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b7346-bfa9-4b70-a3e9-d905d742b44b admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:27.065166 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b7346-bfa9-4b70-a3e9-d905d742b44b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:27.066764 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b7346-bfa9-4b70-a3e9-d905d742b44b admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:27.076200 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1374/2758] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:27.080415 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd32e12-1171-469e-b45f-4966d1faa0b8 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:27.095111 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd32e12-1171-469e-b45f-4966d1faa0b8 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:27.101767 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85de7d-0dcf-4db2-ad1f-ca0a89a2746d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:27.102418 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85de7d-0dcf-4db2-ad1f-ca0a89a2746d admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:27.102700 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85de7d-0dcf-4db2-ad1f-ca0a89a2746d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:27.102976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85de7d-0dcf-4db2-ad1f-ca0a89a2746d admin admin] PATH_INFO: `/v3/projects/78e61b9be06343df871b827751a94eb8/users/b4caed226c284576921fa2252e1035b7/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:27.104898 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85de7d-0dcf-4db2-ad1f-ca0a89a2746d admin admin] RBAC: Authorizing `identity:create_grant(project_id=78e61b9be06343df871b827751a94eb8, user_id=b4caed226c284576921fa2252e1035b7, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:27.108227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85de7d-0dcf-4db2-ad1f-ca0a89a2746d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:27.118605 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1385/2759] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:45:27 2022] PUT /identity/v3/projects/78e61b9be06343df871b827751a94eb8/users/b4caed226c284576921fa2252e1035b7/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:27.130072 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12134c-269d-4aa3-91d1-2b435ee589ba None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:27.130386 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12134c-269d-4aa3-91d1-2b435ee589ba None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:27.130668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12134c-269d-4aa3-91d1-2b435ee589ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:27.158494 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-9e12134c-269d-4aa3-91d1-2b435ee589ba None None] MFA Rules not processed for user `b4caed226c284576921fa2252e1035b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:27.159708 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9e12134c-269d-4aa3-91d1-2b435ee589ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:27.228234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1375/2760] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:27 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:45:27.240031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85de7d-0dcf-4db2-ad1f-ca0a89a2746d admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:27.241748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-2d85de7d-0dcf-4db2-ad1f-ca0a89a2746d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:27.299278 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85de7d-0dcf-4db2-ad1f-ca0a89a2746d admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:27.302870 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e961e7-2806-41db-a828-71a38f7fb4cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:27.303473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e961e7-2806-41db-a828-71a38f7fb4cc service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:27.303752 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e961e7-2806-41db-a828-71a38f7fb4cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:27.304024 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e961e7-2806-41db-a828-71a38f7fb4cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:27.364381 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e961e7-2806-41db-a828-71a38f7fb4cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:27.366182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e961e7-2806-41db-a828-71a38f7fb4cc service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:27.377473 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1386/2761] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:45:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:45:27.449015 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12134c-269d-4aa3-91d1-2b435ee589ba None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:27.450455 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9e12134c-269d-4aa3-91d1-2b435ee589ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:27.519652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12134c-269d-4aa3-91d1-2b435ee589ba None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:27.524888 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0507215-1b96-49f0-af1f-c977934c5cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:27.525556 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0507215-1b96-49f0-af1f-c977934c5cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:27.525834 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0507215-1b96-49f0-af1f-c977934c5cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:27.526112 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0507215-1b96-49f0-af1f-c977934c5cc1 admin admin] PATH_INFO: `/v3/users/6b2ba314d9d44033ab542dec3a0aeff7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:27.527247 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0507215-1b96-49f0-af1f-c977934c5cc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b2ba314d9d44033ab542dec3a0aeff7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:27.529228 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0507215-1b96-49f0-af1f-c977934c5cc1 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:27.817975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e0507215-1b96-49f0-af1f-c977934c5cc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b2ba314d9d44033ab542dec3a0aeff7'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:27.818830 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e0507215-1b96-49f0-af1f-c977934c5cc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b2ba314d9d44033ab542dec3a0aeff7'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:27.922986 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e0507215-1b96-49f0-af1f-c977934c5cc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b2ba314d9d44033ab542dec3a0aeff7'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:27.923627 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e0507215-1b96-49f0-af1f-c977934c5cc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b2ba314d9d44033ab542dec3a0aeff7'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:27.935989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:27.938530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e0507215-1b96-49f0-af1f-c977934c5cc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b2ba314d9d44033ab542dec3a0aeff7'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:27.954654 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1376/2762] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:45:27 2022] DELETE /identity/v3/users/6b2ba314d9d44033ab542dec3a0aeff7 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:27.959275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7a393-a10e-4839-b29b-3205d22d3e6e None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:27.959637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7a393-a10e-4839-b29b-3205d22d3e6e None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:27.959849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7a393-a10e-4839-b29b-3205d22d3e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:27.980123 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-62a7a393-a10e-4839-b29b-3205d22d3e6e None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:27.981506 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-62a7a393-a10e-4839-b29b-3205d22d3e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:28.041798 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1387/2763] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:28.052077 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0507215-1b96-49f0-af1f-c977934c5cc1 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:28.053525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-e0507215-1b96-49f0-af1f-c977934c5cc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:28.111591 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0507215-1b96-49f0-af1f-c977934c5cc1 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:28.115236 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737a4f8-c82d-4c49-aef3-a0319d9e592e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:28.115837 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737a4f8-c82d-4c49-aef3-a0319d9e592e service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:28.116117 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737a4f8-c82d-4c49-aef3-a0319d9e592e service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:28.116389 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737a4f8-c82d-4c49-aef3-a0319d9e592e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:28.126408 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737a4f8-c82d-4c49-aef3-a0319d9e592e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:28.128545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737a4f8-c82d-4c49-aef3-a0319d9e592e service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:28.138520 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1377/2764] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:45:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:45:28.645851 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7a393-a10e-4839-b29b-3205d22d3e6e None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:28.647623 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-62a7a393-a10e-4839-b29b-3205d22d3e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:28.660373 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7a393-a10e-4839-b29b-3205d22d3e6e None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:28.665077 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94fae6-0d75-47f8-a55b-7b2d49755505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:28.665710 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94fae6-0d75-47f8-a55b-7b2d49755505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:28.665985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94fae6-0d75-47f8-a55b-7b2d49755505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:28.666282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94fae6-0d75-47f8-a55b-7b2d49755505 admin admin] PATH_INFO: `/v3/projects/d20926a91920444ca93d2780c90363b5` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:28.667389 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94fae6-0d75-47f8-a55b-7b2d49755505 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d20926a91920444ca93d2780c90363b5)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:28.669405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94fae6-0d75-47f8-a55b-7b2d49755505 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:28.783759 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:28.799029 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-0b94fae6-0d75-47f8-a55b-7b2d49755505 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd20926a91920444ca93d2780c90363b5', 'request_id': 'req-0b94fae6-0d75-47f8-a55b-7b2d49755505'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:28.804479 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-0b94fae6-0d75-47f8-a55b-7b2d49755505 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd20926a91920444ca93d2780c90363b5', 'request_id': 'req-0b94fae6-0d75-47f8-a55b-7b2d49755505'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:28.810655 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-0b94fae6-0d75-47f8-a55b-7b2d49755505 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd20926a91920444ca93d2780c90363b5', 'request_id': 'req-0b94fae6-0d75-47f8-a55b-7b2d49755505'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:28.911897 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-0b94fae6-0d75-47f8-a55b-7b2d49755505 admin admin] The token cache is being invalidate because project d20926a91920444ca93d2780c90363b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:45:28.912213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-0b94fae6-0d75-47f8-a55b-7b2d49755505 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:28.912839 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-0b94fae6-0d75-47f8-a55b-7b2d49755505 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:28.914677 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1388/2765] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:45:28 2022] DELETE /identity/v3/projects/d20926a91920444ca93d2780c90363b5 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:28.919340 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafe569-74e1-44c8-9761-7ca1c7a6c3ee None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:28.920427 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafe569-74e1-44c8-9761-7ca1c7a6c3ee None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:28.920545 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafe569-74e1-44c8-9761-7ca1c7a6c3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:28.940072 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-acafe569-74e1-44c8-9761-7ca1c7a6c3ee None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:28.941304 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-acafe569-74e1-44c8-9761-7ca1c7a6c3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:29.002041 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1378/2766] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:29.009681 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e57f10-5f2d-4cf7-934b-fa640dc78234 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:29.009959 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e57f10-5f2d-4cf7-934b-fa640dc78234 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:29.010249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e57f10-5f2d-4cf7-934b-fa640dc78234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:29.031429 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-85e57f10-5f2d-4cf7-934b-fa640dc78234 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:29.032829 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-85e57f10-5f2d-4cf7-934b-fa640dc78234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:29.041601 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1389/2767] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:29.046220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafe569-74e1-44c8-9761-7ca1c7a6c3ee None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:29.061558 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafe569-74e1-44c8-9761-7ca1c7a6c3ee None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:29.061558 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feebfe9-bd79-4547-b802-442a84d482ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:29.062075 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feebfe9-bd79-4547-b802-442a84d482ac admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:29.062346 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feebfe9-bd79-4547-b802-442a84d482ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:29.062621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feebfe9-bd79-4547-b802-442a84d482ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:29.063605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feebfe9-bd79-4547-b802-442a84d482ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:45:29.063935 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feebfe9-bd79-4547-b802-442a84d482ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:29.065565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feebfe9-bd79-4547-b802-442a84d482ac admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:29.079861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1379/2768] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:45:29.082570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e57f10-5f2d-4cf7-934b-fa640dc78234 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:29.096091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e57f10-5f2d-4cf7-934b-fa640dc78234 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:29.098001 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3ce43-031b-4458-aa31-915881fb61eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:29.098615 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3ce43-031b-4458-aa31-915881fb61eb admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:29.098898 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3ce43-031b-4458-aa31-915881fb61eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:29.099177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3ce43-031b-4458-aa31-915881fb61eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:29.099811 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3ce43-031b-4458-aa31-915881fb61eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:29.101836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3ce43-031b-4458-aa31-915881fb61eb admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:29.211275 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1390/2769] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:45:29 2022] POST /identity/v3/projects => generated 369 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:45:29.215509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feebfe9-bd79-4547-b802-442a84d482ac admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:29.225981 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feebfe9-bd79-4547-b802-442a84d482ac admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:29.285149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d66de-d563-4c10-878a-785f42acbe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:29.285861 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d66de-d563-4c10-878a-785f42acbe32 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:29.286212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d66de-d563-4c10-878a-785f42acbe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:29.286552 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d66de-d563-4c10-878a-785f42acbe32 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:29.287268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d66de-d563-4c10-878a-785f42acbe32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:29.289344 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d66de-d563-4c10-878a-785f42acbe32 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:29.392956 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1380/2770] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:45:29 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:45:29.397158 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3ce43-031b-4458-aa31-915881fb61eb admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:29.407721 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3ce43-031b-4458-aa31-915881fb61eb admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:29.412277 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9691a20-ae32-4427-a467-203ff2ae8106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:29.412869 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9691a20-ae32-4427-a467-203ff2ae8106 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:29.413171 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9691a20-ae32-4427-a467-203ff2ae8106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:29.413444 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9691a20-ae32-4427-a467-203ff2ae8106 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:29.414083 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9691a20-ae32-4427-a467-203ff2ae8106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:29.415667 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9691a20-ae32-4427-a467-203ff2ae8106 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:29.426952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1391/2771] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:29.430032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d66de-d563-4c10-878a-785f42acbe32 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:29.438116 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:29.442041 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d66de-d563-4c10-878a-785f42acbe32 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:29.446814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebd613-07be-4b59-9212-d0d8cd451f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:29.447349 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebd613-07be-4b59-9212-d0d8cd451f2d admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:29.447630 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebd613-07be-4b59-9212-d0d8cd451f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:29.447905 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebd613-07be-4b59-9212-d0d8cd451f2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:29.448550 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebd613-07be-4b59-9212-d0d8cd451f2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:29.450151 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebd613-07be-4b59-9212-d0d8cd451f2d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:29.460475 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1381/2772] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:29.464820 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9691a20-ae32-4427-a467-203ff2ae8106 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:29.475836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9691a20-ae32-4427-a467-203ff2ae8106 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:29.487527 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08799cfc-cc16-42ae-a97f-22a8d99b610c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:29.488477 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08799cfc-cc16-42ae-a97f-22a8d99b610c admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:29.488759 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08799cfc-cc16-42ae-a97f-22a8d99b610c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:29.489034 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08799cfc-cc16-42ae-a97f-22a8d99b610c admin admin] PATH_INFO: `/v3/projects/e12d80e5675745379eba8b615fb61048/users/e264aeff7a904b03bf3116c5b52620b0/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:29.600417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebd613-07be-4b59-9212-d0d8cd451f2d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:29.602300 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-53ebd613-07be-4b59-9212-d0d8cd451f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:29.632702 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08799cfc-cc16-42ae-a97f-22a8d99b610c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e12d80e5675745379eba8b615fb61048, user_id=e264aeff7a904b03bf3116c5b52620b0, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:29.635417 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08799cfc-cc16-42ae-a97f-22a8d99b610c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:29.661444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebd613-07be-4b59-9212-d0d8cd451f2d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:29.665039 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcbdf3-d59b-4d79-b5f8-dfd76f97b66f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:29.665662 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcbdf3-d59b-4d79-b5f8-dfd76f97b66f service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:29.665940 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcbdf3-d59b-4d79-b5f8-dfd76f97b66f service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:29.666214 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcbdf3-d59b-4d79-b5f8-dfd76f97b66f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:29.667937 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d8bcbdf3-d59b-4d79-b5f8-dfd76f97b66f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:29.729007 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcbdf3-d59b-4d79-b5f8-dfd76f97b66f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:29.730506 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcbdf3-d59b-4d79-b5f8-dfd76f97b66f service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:29.739914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1382/2773] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:45:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:45:29.747538 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1392/2774] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:45:29 2022] PUT /identity/v3/projects/e12d80e5675745379eba8b615fb61048/users/e264aeff7a904b03bf3116c5b52620b0/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:29.759323 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bc0a1-4d59-4885-b299-422382bd57e2 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:29.759613 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bc0a1-4d59-4885-b299-422382bd57e2 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:29.759893 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bc0a1-4d59-4885-b299-422382bd57e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:29.788161 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-bd7bc0a1-4d59-4885-b299-422382bd57e2 None None] MFA Rules not processed for user `e264aeff7a904b03bf3116c5b52620b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:29.789434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bd7bc0a1-4d59-4885-b299-422382bd57e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:29.854338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1383/2775] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:29.868622 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08799cfc-cc16-42ae-a97f-22a8d99b610c admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:29.870449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-08799cfc-cc16-42ae-a97f-22a8d99b610c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:29.932599 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08799cfc-cc16-42ae-a97f-22a8d99b610c admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:29.936314 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7743e0f-ea0f-4e4e-a074-c39fa3095947 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a40b1ed93a6b4e5fb0e8f907627379a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:29.936927 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7743e0f-ea0f-4e4e-a074-c39fa3095947 service glance] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:29.937242 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7743e0f-ea0f-4e4e-a074-c39fa3095947 service glance] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:29.946670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7743e0f-ea0f-4e4e-a074-c39fa3095947 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:29.946670 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7743e0f-ea0f-4e4e-a074-c39fa3095947 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:29.947546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7743e0f-ea0f-4e4e-a074-c39fa3095947 service glance] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:29.956776 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1393/2776] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:45:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:45:31.780562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7bc0a1-4d59-4885-b299-422382bd57e2 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:31.823971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=100 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:31.838762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7bc0a1-4d59-4885-b299-422382bd57e2 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:31.842400 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f852e8-8ff2-43c2-89ce-89e972c2fae6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:31.843002 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f852e8-8ff2-43c2-89ce-89e972c2fae6 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:31.843282 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f852e8-8ff2-43c2-89ce-89e972c2fae6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:31.843560 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f852e8-8ff2-43c2-89ce-89e972c2fae6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:31.845422 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d0f852e8-8ff2-43c2-89ce-89e972c2fae6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:31.910721 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f852e8-8ff2-43c2-89ce-89e972c2fae6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:31.912145 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f852e8-8ff2-43c2-89ce-89e972c2fae6 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:31.921554 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1384/2777] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:45:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:45:33.750922 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7743e0f-ea0f-4e4e-a074-c39fa3095947 service glance] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:33.752512 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c7743e0f-ea0f-4e4e-a074-c39fa3095947 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:33.816928 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:33.859003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7743e0f-ea0f-4e4e-a074-c39fa3095947 service glance] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:33.864497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de2813-bc3e-4e8b-9ea5-bbb879608b69 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0ef47a60f54ca1853fd267e308c58b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:33.865188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de2813-bc3e-4e8b-9ea5-bbb879608b69 service placement] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:33.865422 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de2813-bc3e-4e8b-9ea5-bbb879608b69 service placement] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:33.865677 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de2813-bc3e-4e8b-9ea5-bbb879608b69 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:33.868442 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e4de2813-bc3e-4e8b-9ea5-bbb879608b69 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:33.882027 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de2813-bc3e-4e8b-9ea5-bbb879608b69 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:33.883342 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de2813-bc3e-4e8b-9ea5-bbb879608b69 service placement] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:33.894103 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1394/2778] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:45:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:45:40.113817 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f852e8-8ff2-43c2-89ce-89e972c2fae6 service cinder] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:40.115342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-d0f852e8-8ff2-43c2-89ce-89e972c2fae6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:40.183746 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f852e8-8ff2-43c2-89ce-89e972c2fae6 service cinder] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:40.188776 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471dea0f-5ec9-45ad-807f-033abbc089e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:40.189490 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dea0f-5ec9-45ad-807f-033abbc089e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:40.189769 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dea0f-5ec9-45ad-807f-033abbc089e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:40.190041 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dea0f-5ec9-45ad-807f-033abbc089e3 admin admin] PATH_INFO: `/v3/users/7dab6b337b5a4124afece3118ff3d51e` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:40.191169 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471dea0f-5ec9-45ad-807f-033abbc089e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dab6b337b5a4124afece3118ff3d51e)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:40.193143 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471dea0f-5ec9-45ad-807f-033abbc089e3 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:40.400761 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-471dea0f-5ec9-45ad-807f-033abbc089e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dab6b337b5a4124afece3118ff3d51e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:40.401389 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-471dea0f-5ec9-45ad-807f-033abbc089e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dab6b337b5a4124afece3118ff3d51e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:40.478193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-471dea0f-5ec9-45ad-807f-033abbc089e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dab6b337b5a4124afece3118ff3d51e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:40.478819 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-471dea0f-5ec9-45ad-807f-033abbc089e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dab6b337b5a4124afece3118ff3d51e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:40.491964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-471dea0f-5ec9-45ad-807f-033abbc089e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dab6b337b5a4124afece3118ff3d51e'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:40.504163 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1385/2779] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:45:40 2022] DELETE /identity/v3/users/7dab6b337b5a4124afece3118ff3d51e => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:40.508913 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bfffa-76a9-43f7-ae72-e0103eec6cd8 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:40.509403 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bfffa-76a9-43f7-ae72-e0103eec6cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:40.509502 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bfffa-76a9-43f7-ae72-e0103eec6cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:40.529432 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-5f9bfffa-76a9-43f7-ae72-e0103eec6cd8 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:40.530697 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5f9bfffa-76a9-43f7-ae72-e0103eec6cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:40.590186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1395/2780] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:40.600633 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471dea0f-5ec9-45ad-807f-033abbc089e3 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:40.602015 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-471dea0f-5ec9-45ad-807f-033abbc089e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:40.611929 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:40.659840 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471dea0f-5ec9-45ad-807f-033abbc089e3 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:40.663563 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed939164-bf40-48f4-8f48-6db1a71ff76b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:40.664216 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed939164-bf40-48f4-8f48-6db1a71ff76b service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:40.664511 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed939164-bf40-48f4-8f48-6db1a71ff76b service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:40.664781 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed939164-bf40-48f4-8f48-6db1a71ff76b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:40.674235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed939164-bf40-48f4-8f48-6db1a71ff76b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:40.677291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed939164-bf40-48f4-8f48-6db1a71ff76b service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:40.684974 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1386/2781] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:45:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:45:41.154272 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bfffa-76a9-43f7-ae72-e0103eec6cd8 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:41.155964 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5f9bfffa-76a9-43f7-ae72-e0103eec6cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:41.178917 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bfffa-76a9-43f7-ae72-e0103eec6cd8 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:41.181171 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a92172-1ba7-4a94-b1ed-76aae03c8fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:41.181916 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a92172-1ba7-4a94-b1ed-76aae03c8fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:41.182253 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a92172-1ba7-4a94-b1ed-76aae03c8fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:41.182587 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a92172-1ba7-4a94-b1ed-76aae03c8fce admin admin] PATH_INFO: `/v3/projects/fe82ada6f5f34dac92c3df01a052a827` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:41.183889 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a92172-1ba7-4a94-b1ed-76aae03c8fce admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe82ada6f5f34dac92c3df01a052a827)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:41.186163 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a92172-1ba7-4a94-b1ed-76aae03c8fce admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:41.337086 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-74a92172-1ba7-4a94-b1ed-76aae03c8fce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe82ada6f5f34dac92c3df01a052a827', 'request_id': 'req-74a92172-1ba7-4a94-b1ed-76aae03c8fce'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:41.342903 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-74a92172-1ba7-4a94-b1ed-76aae03c8fce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe82ada6f5f34dac92c3df01a052a827', 'request_id': 'req-74a92172-1ba7-4a94-b1ed-76aae03c8fce'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:41.352692 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-74a92172-1ba7-4a94-b1ed-76aae03c8fce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe82ada6f5f34dac92c3df01a052a827', 'request_id': 'req-74a92172-1ba7-4a94-b1ed-76aae03c8fce'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:41.518433 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-74a92172-1ba7-4a94-b1ed-76aae03c8fce admin admin] The token cache is being invalidate because project fe82ada6f5f34dac92c3df01a052a827 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:45:41.519137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-74a92172-1ba7-4a94-b1ed-76aae03c8fce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:41.520218 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-74a92172-1ba7-4a94-b1ed-76aae03c8fce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:41.521856 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1396/2782] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:45:41 2022] DELETE /identity/v3/projects/fe82ada6f5f34dac92c3df01a052a827 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:41.526174 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed939164-bf40-48f4-8f48-6db1a71ff76b service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:41.538568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed939164-bf40-48f4-8f48-6db1a71ff76b service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:41.596937 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade12ce0-98c4-41c6-a9af-39c78ed29274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:41.597613 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade12ce0-98c4-41c6-a9af-39c78ed29274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:41.597878 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade12ce0-98c4-41c6-a9af-39c78ed29274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:41.598154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade12ce0-98c4-41c6-a9af-39c78ed29274 admin admin] PATH_INFO: `/v3/users/618a9960f0f5485bb9d4cd88651fd767` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:41.599274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade12ce0-98c4-41c6-a9af-39c78ed29274 admin admin] RBAC: Authorizing `identity:delete_user(user_id=618a9960f0f5485bb9d4cd88651fd767)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:41.601414 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade12ce0-98c4-41c6-a9af-39c78ed29274 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:41.844302 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ade12ce0-98c4-41c6-a9af-39c78ed29274 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '618a9960f0f5485bb9d4cd88651fd767'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:41.845016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ade12ce0-98c4-41c6-a9af-39c78ed29274 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '618a9960f0f5485bb9d4cd88651fd767'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:41.859506 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a92172-1ba7-4a94-b1ed-76aae03c8fce admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:41.860959 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-74a92172-1ba7-4a94-b1ed-76aae03c8fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:41.874469 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a92172-1ba7-4a94-b1ed-76aae03c8fce admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:41.879995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926af154-0662-4e3a-8fcb-e1befe94e83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:41.880637 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926af154-0662-4e3a-8fcb-e1befe94e83e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:41.880916 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926af154-0662-4e3a-8fcb-e1befe94e83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:41.881224 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926af154-0662-4e3a-8fcb-e1befe94e83e admin admin] PATH_INFO: `/v3/users/33b40edf99124598b633ff30b4e31a14` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:41.883954 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926af154-0662-4e3a-8fcb-e1befe94e83e admin admin] RBAC: Authorizing `identity:delete_user(user_id=33b40edf99124598b633ff30b4e31a14)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:41.884245 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926af154-0662-4e3a-8fcb-e1befe94e83e admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:41.906895 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ade12ce0-98c4-41c6-a9af-39c78ed29274 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '618a9960f0f5485bb9d4cd88651fd767'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:41.907518 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ade12ce0-98c4-41c6-a9af-39c78ed29274 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '618a9960f0f5485bb9d4cd88651fd767'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:41.920944 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ade12ce0-98c4-41c6-a9af-39c78ed29274 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '618a9960f0f5485bb9d4cd88651fd767'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:41.934123 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1387/2783] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:45:41 2022] DELETE /identity/v3/users/618a9960f0f5485bb9d4cd88651fd767 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:45:42.044851 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:42.047427 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-926af154-0662-4e3a-8fcb-e1befe94e83e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33b40edf99124598b633ff30b4e31a14'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:42.147962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-926af154-0662-4e3a-8fcb-e1befe94e83e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33b40edf99124598b633ff30b4e31a14'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:42.148854 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-926af154-0662-4e3a-8fcb-e1befe94e83e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33b40edf99124598b633ff30b4e31a14'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:42.149784 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-926af154-0662-4e3a-8fcb-e1befe94e83e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33b40edf99124598b633ff30b4e31a14'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:42.169969 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-926af154-0662-4e3a-8fcb-e1befe94e83e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33b40edf99124598b633ff30b4e31a14'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:42.204496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1397/2784] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:45:41 2022] DELETE /identity/v3/users/33b40edf99124598b633ff30b4e31a14 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:45:42.208908 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f158b1b-3fb4-4c4a-bca5-76fe3ed5dd77 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:42.209561 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f158b1b-3fb4-4c4a-bca5-76fe3ed5dd77 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:42.209941 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f158b1b-3fb4-4c4a-bca5-76fe3ed5dd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:42.234928 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-0f158b1b-3fb4-4c4a-bca5-76fe3ed5dd77 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:42.236668 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0f158b1b-3fb4-4c4a-bca5-76fe3ed5dd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:42.269249 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:42.305841 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1388/2785] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:42.316295 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926af154-0662-4e3a-8fcb-e1befe94e83e admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:42.318092 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-926af154-0662-4e3a-8fcb-e1befe94e83e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:42.387133 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926af154-0662-4e3a-8fcb-e1befe94e83e admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:42.390775 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfc4e3-da77-45af-9e1d-0dd86861806c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:42.391399 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfc4e3-da77-45af-9e1d-0dd86861806c service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:42.391673 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfc4e3-da77-45af-9e1d-0dd86861806c service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:42.391942 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfc4e3-da77-45af-9e1d-0dd86861806c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:42.401380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfc4e3-da77-45af-9e1d-0dd86861806c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:42.408516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfc4e3-da77-45af-9e1d-0dd86861806c service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:42.412621 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1398/2786] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:45:42.440212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f158b1b-3fb4-4c4a-bca5-76fe3ed5dd77 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:42.441685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0f158b1b-3fb4-4c4a-bca5-76fe3ed5dd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:42.456401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f158b1b-3fb4-4c4a-bca5-76fe3ed5dd77 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:42.466858 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff290ee-a906-45ad-a2bf-5d359db61cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:42.471565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff290ee-a906-45ad-a2bf-5d359db61cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:42.471565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff290ee-a906-45ad-a2bf-5d359db61cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:42.471565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff290ee-a906-45ad-a2bf-5d359db61cc2 admin admin] PATH_INFO: `/v3/projects/1a7bac3c4746437abd9041203e42fed8` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:42.471565 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff290ee-a906-45ad-a2bf-5d359db61cc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a7bac3c4746437abd9041203e42fed8)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:42.474748 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff290ee-a906-45ad-a2bf-5d359db61cc2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:42.593826 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ff290ee-a906-45ad-a2bf-5d359db61cc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a7bac3c4746437abd9041203e42fed8', 'request_id': 'req-6ff290ee-a906-45ad-a2bf-5d359db61cc2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:42.601097 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ff290ee-a906-45ad-a2bf-5d359db61cc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a7bac3c4746437abd9041203e42fed8', 'request_id': 'req-6ff290ee-a906-45ad-a2bf-5d359db61cc2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:42.609026 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ff290ee-a906-45ad-a2bf-5d359db61cc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a7bac3c4746437abd9041203e42fed8', 'request_id': 'req-6ff290ee-a906-45ad-a2bf-5d359db61cc2'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:42.735135 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ff290ee-a906-45ad-a2bf-5d359db61cc2 admin admin] The token cache is being invalidate because project 1a7bac3c4746437abd9041203e42fed8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:45:42.735580 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ff290ee-a906-45ad-a2bf-5d359db61cc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:42.736215 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ff290ee-a906-45ad-a2bf-5d359db61cc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:42.737853 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1389/2787] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:45:42 2022] DELETE /identity/v3/projects/1a7bac3c4746437abd9041203e42fed8 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:42.742685 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfc4e3-da77-45af-9e1d-0dd86861806c service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:42.753294 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfc4e3-da77-45af-9e1d-0dd86861806c service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:42.815450 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec0e82-182c-4b40-b46c-18bdaf773eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:42.816079 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec0e82-182c-4b40-b46c-18bdaf773eab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:42.816355 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec0e82-182c-4b40-b46c-18bdaf773eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:42.816763 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec0e82-182c-4b40-b46c-18bdaf773eab admin admin] PATH_INFO: `/v3/users/b4caed226c284576921fa2252e1035b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:42.817977 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec0e82-182c-4b40-b46c-18bdaf773eab admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4caed226c284576921fa2252e1035b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:42.821749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec0e82-182c-4b40-b46c-18bdaf773eab admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:42.868434 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff290ee-a906-45ad-a2bf-5d359db61cc2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:42.870087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6ff290ee-a906-45ad-a2bf-5d359db61cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:42.883245 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff290ee-a906-45ad-a2bf-5d359db61cc2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:42.888011 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe2a72-0324-49b0-8d0e-dc669d468eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:42.888700 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe2a72-0324-49b0-8d0e-dc669d468eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:42.892915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe2a72-0324-49b0-8d0e-dc669d468eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:42.892915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe2a72-0324-49b0-8d0e-dc669d468eba admin admin] PATH_INFO: `/v3/projects/af021dbd14a5406ab91700845ee808ef` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:42.892915 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe2a72-0324-49b0-8d0e-dc669d468eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=af021dbd14a5406ab91700845ee808ef)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:42.893346 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe2a72-0324-49b0-8d0e-dc669d468eba admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:43.127643 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a8fe2a72-0324-49b0-8d0e-dc669d468eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af021dbd14a5406ab91700845ee808ef', 'request_id': 'req-a8fe2a72-0324-49b0-8d0e-dc669d468eba'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:43.129218 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-57ec0e82-182c-4b40-b46c-18bdaf773eab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4caed226c284576921fa2252e1035b7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:43.134056 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a8fe2a72-0324-49b0-8d0e-dc669d468eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af021dbd14a5406ab91700845ee808ef', 'request_id': 'req-a8fe2a72-0324-49b0-8d0e-dc669d468eba'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:43.142957 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a8fe2a72-0324-49b0-8d0e-dc669d468eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af021dbd14a5406ab91700845ee808ef', 'request_id': 'req-a8fe2a72-0324-49b0-8d0e-dc669d468eba'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:43.242955 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-57ec0e82-182c-4b40-b46c-18bdaf773eab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4caed226c284576921fa2252e1035b7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:43.244066 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-57ec0e82-182c-4b40-b46c-18bdaf773eab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4caed226c284576921fa2252e1035b7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:43.244741 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-57ec0e82-182c-4b40-b46c-18bdaf773eab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4caed226c284576921fa2252e1035b7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:43.253935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-57ec0e82-182c-4b40-b46c-18bdaf773eab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4caed226c284576921fa2252e1035b7'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:43.255808 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:43.272009 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1399/2788] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:45:42 2022] DELETE /identity/v3/users/b4caed226c284576921fa2252e1035b7 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:43.318008 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a8fe2a72-0324-49b0-8d0e-dc669d468eba admin admin] The token cache is being invalidate because project af021dbd14a5406ab91700845ee808ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:45:43.318303 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a8fe2a72-0324-49b0-8d0e-dc669d468eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:43.318977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-a8fe2a72-0324-49b0-8d0e-dc669d468eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:43.320509 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1390/2789] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:45:42 2022] DELETE /identity/v3/projects/af021dbd14a5406ab91700845ee808ef => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:45:43.329885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec0e82-182c-4b40-b46c-18bdaf773eab admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:43.331569 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-57ec0e82-182c-4b40-b46c-18bdaf773eab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:43.406604 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec0e82-182c-4b40-b46c-18bdaf773eab admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:43.411235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9b7af-bea0-483c-8273-56b013681ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:43.411972 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9b7af-bea0-483c-8273-56b013681ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:43.412250 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9b7af-bea0-483c-8273-56b013681ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:43.412518 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9b7af-bea0-483c-8273-56b013681ad7 admin admin] PATH_INFO: `/v3/users/b5f5ce09258f4ea383040bea1bddcdb1` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:43.413651 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9b7af-bea0-483c-8273-56b013681ad7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5f5ce09258f4ea383040bea1bddcdb1)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:43.415914 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9b7af-bea0-483c-8273-56b013681ad7 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:43.609355 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-3df9b7af-bea0-483c-8273-56b013681ad7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5f5ce09258f4ea383040bea1bddcdb1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:43.707881 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe2a72-0324-49b0-8d0e-dc669d468eba admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:43.709360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a8fe2a72-0324-49b0-8d0e-dc669d468eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:43.734479 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-3df9b7af-bea0-483c-8273-56b013681ad7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5f5ce09258f4ea383040bea1bddcdb1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:43.734479 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-3df9b7af-bea0-483c-8273-56b013681ad7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5f5ce09258f4ea383040bea1bddcdb1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:43.734830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-3df9b7af-bea0-483c-8273-56b013681ad7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5f5ce09258f4ea383040bea1bddcdb1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:43.743681 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-3df9b7af-bea0-483c-8273-56b013681ad7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5f5ce09258f4ea383040bea1bddcdb1'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:43.762754 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:43.764597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1400/2790] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:45:43 2022] DELETE /identity/v3/users/b5f5ce09258f4ea383040bea1bddcdb1 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:43.790771 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe2a72-0324-49b0-8d0e-dc669d468eba admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:43.863207 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887026b-ccb8-47e9-8aae-0f387b2c60ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:43.863863 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887026b-ccb8-47e9-8aae-0f387b2c60ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:43.864172 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887026b-ccb8-47e9-8aae-0f387b2c60ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:43.864451 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887026b-ccb8-47e9-8aae-0f387b2c60ae admin admin] PATH_INFO: `/v3/projects/78e61b9be06343df871b827751a94eb8` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:43.865621 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887026b-ccb8-47e9-8aae-0f387b2c60ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=78e61b9be06343df871b827751a94eb8)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:43.867589 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887026b-ccb8-47e9-8aae-0f387b2c60ae admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:43.983530 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e887026b-ccb8-47e9-8aae-0f387b2c60ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78e61b9be06343df871b827751a94eb8', 'request_id': 'req-e887026b-ccb8-47e9-8aae-0f387b2c60ae'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:43.990719 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e887026b-ccb8-47e9-8aae-0f387b2c60ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78e61b9be06343df871b827751a94eb8', 'request_id': 'req-e887026b-ccb8-47e9-8aae-0f387b2c60ae'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:43.997178 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e887026b-ccb8-47e9-8aae-0f387b2c60ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78e61b9be06343df871b827751a94eb8', 'request_id': 'req-e887026b-ccb8-47e9-8aae-0f387b2c60ae'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:44.134637 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e887026b-ccb8-47e9-8aae-0f387b2c60ae admin admin] The token cache is being invalidate because project 78e61b9be06343df871b827751a94eb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:45:44.134940 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e887026b-ccb8-47e9-8aae-0f387b2c60ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:44.135568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e887026b-ccb8-47e9-8aae-0f387b2c60ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:44.137220 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1391/2791] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:45:43 2022] DELETE /identity/v3/projects/78e61b9be06343df871b827751a94eb8 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:45:44.269166 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9b7af-bea0-483c-8273-56b013681ad7 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:44.271539 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-3df9b7af-bea0-483c-8273-56b013681ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:44.289898 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9b7af-bea0-483c-8273-56b013681ad7 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:44.364227 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc8b4d-b734-421f-a25e-79850ea41e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:44.364868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc8b4d-b734-421f-a25e-79850ea41e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:44.365195 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc8b4d-b734-421f-a25e-79850ea41e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:44.365466 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc8b4d-b734-421f-a25e-79850ea41e66 admin admin] PATH_INFO: `/v3/projects/5eb8972b0a4e4a128845a1350e9ec590` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:44.366691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc8b4d-b734-421f-a25e-79850ea41e66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eb8972b0a4e4a128845a1350e9ec590)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:44.368710 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc8b4d-b734-421f-a25e-79850ea41e66 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:44.371521 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:44.495553 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2afc8b4d-b734-421f-a25e-79850ea41e66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eb8972b0a4e4a128845a1350e9ec590', 'request_id': 'req-2afc8b4d-b734-421f-a25e-79850ea41e66'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:44.501031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2afc8b4d-b734-421f-a25e-79850ea41e66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eb8972b0a4e4a128845a1350e9ec590', 'request_id': 'req-2afc8b4d-b734-421f-a25e-79850ea41e66'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:44.507870 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2afc8b4d-b734-421f-a25e-79850ea41e66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eb8972b0a4e4a128845a1350e9ec590', 'request_id': 'req-2afc8b4d-b734-421f-a25e-79850ea41e66'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:44.592389 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2afc8b4d-b734-421f-a25e-79850ea41e66 admin admin] The token cache is being invalidate because project 5eb8972b0a4e4a128845a1350e9ec590 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:45:44.592684 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2afc8b4d-b734-421f-a25e-79850ea41e66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:44.593282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2afc8b4d-b734-421f-a25e-79850ea41e66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:44.594950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1401/2792] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:45:44 2022] DELETE /identity/v3/projects/5eb8972b0a4e4a128845a1350e9ec590 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:44.600664 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a531967-d877-4f6c-bfa2-d47a20fcb3e0 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:44.600951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a531967-d877-4f6c-bfa2-d47a20fcb3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:44.601267 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a531967-d877-4f6c-bfa2-d47a20fcb3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:44.621464 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-9a531967-d877-4f6c-bfa2-d47a20fcb3e0 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:44.622769 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9a531967-d877-4f6c-bfa2-d47a20fcb3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:44.685470 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1392/2793] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:44.694640 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6b3c4-a1ce-46bf-a451-113919a828ab None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:44.694927 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6b3c4-a1ce-46bf-a451-113919a828ab None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:44.695207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6b3c4-a1ce-46bf-a451-113919a828ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:44.715490 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-97c6b3c4-a1ce-46bf-a451-113919a828ab None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:44.716724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-97c6b3c4-a1ce-46bf-a451-113919a828ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:44.728037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1402/2794] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:44.730094 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a531967-d877-4f6c-bfa2-d47a20fcb3e0 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:44.740387 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a531967-d877-4f6c-bfa2-d47a20fcb3e0 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:44.745030 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8777e16-c83d-4cc5-89fb-99e4fa106f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:44.745653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8777e16-c83d-4cc5-89fb-99e4fa106f72 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:44.745934 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8777e16-c83d-4cc5-89fb-99e4fa106f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:44.746213 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8777e16-c83d-4cc5-89fb-99e4fa106f72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:44.746860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8777e16-c83d-4cc5-89fb-99e4fa106f72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:45:44.747167 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8777e16-c83d-4cc5-89fb-99e4fa106f72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:44.748755 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8777e16-c83d-4cc5-89fb-99e4fa106f72 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:44.757525 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:44.763652 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1393/2795] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:45:44.765372 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6b3c4-a1ce-46bf-a451-113919a828ab None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:44.775110 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6b3c4-a1ce-46bf-a451-113919a828ab None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:44.779784 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e102ed5-c57d-459f-b652-c217743a140b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:44.780380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e102ed5-c57d-459f-b652-c217743a140b admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:44.780658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e102ed5-c57d-459f-b652-c217743a140b admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:44.780929 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e102ed5-c57d-459f-b652-c217743a140b admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:44.781665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e102ed5-c57d-459f-b652-c217743a140b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:44.784130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e102ed5-c57d-459f-b652-c217743a140b admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:45.136896 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1403/2796] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:45:44 2022] POST /identity/v3/projects => generated 373 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:45:45.141284 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8777e16-c83d-4cc5-89fb-99e4fa106f72 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:45.152029 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8777e16-c83d-4cc5-89fb-99e4fa106f72 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:45.210245 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cedb8-b9b5-4035-9d1a-879fbaa4d59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:45.210863 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cedb8-b9b5-4035-9d1a-879fbaa4d59c admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:45.211139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cedb8-b9b5-4035-9d1a-879fbaa4d59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:45.211407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cedb8-b9b5-4035-9d1a-879fbaa4d59c admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:45.212042 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cedb8-b9b5-4035-9d1a-879fbaa4d59c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:45.214035 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cedb8-b9b5-4035-9d1a-879fbaa4d59c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:45.307326 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1394/2797] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:45:45 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:45:45.311616 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e102ed5-c57d-459f-b652-c217743a140b admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:45.322428 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e102ed5-c57d-459f-b652-c217743a140b admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:45.326965 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0f5d0-e1b7-4095-8628-288fd6d6201a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:45.327556 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0f5d0-e1b7-4095-8628-288fd6d6201a admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:45.327830 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0f5d0-e1b7-4095-8628-288fd6d6201a admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:45.328097 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0f5d0-e1b7-4095-8628-288fd6d6201a admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:45.328721 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0f5d0-e1b7-4095-8628-288fd6d6201a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:45.330925 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0f5d0-e1b7-4095-8628-288fd6d6201a admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:45.340742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1404/2798] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:45.345719 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cedb8-b9b5-4035-9d1a-879fbaa4d59c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:45.356941 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cedb8-b9b5-4035-9d1a-879fbaa4d59c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:45.361657 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fde9e9-2a54-4cdb-ac8f-c3465d71d54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:45.362264 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fde9e9-2a54-4cdb-ac8f-c3465d71d54c admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:45.362550 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fde9e9-2a54-4cdb-ac8f-c3465d71d54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:45.362830 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fde9e9-2a54-4cdb-ac8f-c3465d71d54c admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:45.363476 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fde9e9-2a54-4cdb-ac8f-c3465d71d54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:45.365091 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fde9e9-2a54-4cdb-ac8f-c3465d71d54c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:45.377311 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1395/2799] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:45.382193 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0f5d0-e1b7-4095-8628-288fd6d6201a admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:45.393598 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0f5d0-e1b7-4095-8628-288fd6d6201a admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:45.398851 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4ca3a-36e4-4791-858a-e7c99529e623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:45.399599 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4ca3a-36e4-4791-858a-e7c99529e623 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:45.399881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4ca3a-36e4-4791-858a-e7c99529e623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:45.400167 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4ca3a-36e4-4791-858a-e7c99529e623 admin admin] PATH_INFO: `/v3/projects/5ed5f8c3f795487dacc9b2c005118d84/users/d10406d49fe149b7a486afa5d9627a27/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:45.415184 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4ca3a-36e4-4791-858a-e7c99529e623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ed5f8c3f795487dacc9b2c005118d84, user_id=d10406d49fe149b7a486afa5d9627a27, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:45.418094 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4ca3a-36e4-4791-858a-e7c99529e623 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:45.541846 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1405/2800] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:45:45 2022] PUT /identity/v3/projects/5ed5f8c3f795487dacc9b2c005118d84/users/d10406d49fe149b7a486afa5d9627a27/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:45.553386 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109184f-6ed1-41f3-a058-08d960128d99 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:45.553712 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109184f-6ed1-41f3-a058-08d960128d99 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:45.554020 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109184f-6ed1-41f3-a058-08d960128d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:45.586263 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-4109184f-6ed1-41f3-a058-08d960128d99 None None] MFA Rules not processed for user `d10406d49fe149b7a486afa5d9627a27`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:45.587532 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-4109184f-6ed1-41f3-a058-08d960128d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:45.653935 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1396/2801] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:45.658652 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4ca3a-36e4-4791-858a-e7c99529e623 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:45.720079 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4ca3a-36e4-4791-858a-e7c99529e623 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:45.725249 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc3bb2-09a9-480b-9c59-e94e43fd5eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:45.725864 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc3bb2-09a9-480b-9c59-e94e43fd5eac admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:45.726145 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc3bb2-09a9-480b-9c59-e94e43fd5eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:45.726420 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc3bb2-09a9-480b-9c59-e94e43fd5eac admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:45.727054 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc3bb2-09a9-480b-9c59-e94e43fd5eac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:45.728963 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc3bb2-09a9-480b-9c59-e94e43fd5eac admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:45.856748 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1406/2802] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:45:45 2022] POST /identity/v3/projects => generated 375 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:45:45.861544 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109184f-6ed1-41f3-a058-08d960128d99 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:45.872158 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109184f-6ed1-41f3-a058-08d960128d99 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:45.935217 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733d7a5-7e5c-4db1-896f-75cc52586bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:45.935865 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733d7a5-7e5c-4db1-896f-75cc52586bdf admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:45.936167 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733d7a5-7e5c-4db1-896f-75cc52586bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:45.936459 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733d7a5-7e5c-4db1-896f-75cc52586bdf admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:45.937201 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1733d7a5-7e5c-4db1-896f-75cc52586bdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:45.940013 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1733d7a5-7e5c-4db1-896f-75cc52586bdf admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:46.038464 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:46.045005 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1397/2803] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:45:45 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:45:46.048893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc3bb2-09a9-480b-9c59-e94e43fd5eac admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:46.060375 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc3bb2-09a9-480b-9c59-e94e43fd5eac admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:46.065921 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d209307-8281-4b8a-8630-f3a358c9dcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:46.066535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d209307-8281-4b8a-8630-f3a358c9dcd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:46.066817 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d209307-8281-4b8a-8630-f3a358c9dcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:46.067093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d209307-8281-4b8a-8630-f3a358c9dcd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:46.067772 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d209307-8281-4b8a-8630-f3a358c9dcd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:46.069392 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d209307-8281-4b8a-8630-f3a358c9dcd2 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:46.082219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1407/2804] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:46.086444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733d7a5-7e5c-4db1-896f-75cc52586bdf admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:46.097647 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733d7a5-7e5c-4db1-896f-75cc52586bdf admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:46.103082 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3d418-6567-4292-af07-47174b13c578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:46.103780 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3d418-6567-4292-af07-47174b13c578 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:46.104072 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3d418-6567-4292-af07-47174b13c578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:46.104335 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3d418-6567-4292-af07-47174b13c578 admin admin] PATH_INFO: `/v3/projects/1cf755f2acd146ef9f21fc1d4ddf1df8/users/46c8c9c636d84b5cbde3151f3733ee75/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:46.120514 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3d418-6567-4292-af07-47174b13c578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf755f2acd146ef9f21fc1d4ddf1df8, user_id=46c8c9c636d84b5cbde3151f3733ee75, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:46.123767 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3d418-6567-4292-af07-47174b13c578 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:46.203797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1398/2805] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:45:46 2022] PUT /identity/v3/projects/1cf755f2acd146ef9f21fc1d4ddf1df8/users/46c8c9c636d84b5cbde3151f3733ee75/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:45:46.208384 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d209307-8281-4b8a-8630-f3a358c9dcd2 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:46.213482 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:46.218925 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d209307-8281-4b8a-8630-f3a358c9dcd2 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:46.273895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d366e9-b9b0-4f66-9d49-40c283c31843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:46.274490 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d366e9-b9b0-4f66-9d49-40c283c31843 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:46.274764 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d366e9-b9b0-4f66-9d49-40c283c31843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:46.275035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d366e9-b9b0-4f66-9d49-40c283c31843 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:46.275661 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d366e9-b9b0-4f66-9d49-40c283c31843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:46.277282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d366e9-b9b0-4f66-9d49-40c283c31843 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:46.292260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1408/2806] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:46.297048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3d418-6567-4292-af07-47174b13c578 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:46.309701 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3d418-6567-4292-af07-47174b13c578 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:46.314977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67eebc-ee16-4144-bc5e-54567c237c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:46.315635 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67eebc-ee16-4144-bc5e-54567c237c67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:46.316028 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67eebc-ee16-4144-bc5e-54567c237c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:46.316311 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67eebc-ee16-4144-bc5e-54567c237c67 admin admin] PATH_INFO: `/v3/projects/1cf755f2acd146ef9f21fc1d4ddf1df8/users/46c8c9c636d84b5cbde3151f3733ee75/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:46.318367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67eebc-ee16-4144-bc5e-54567c237c67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf755f2acd146ef9f21fc1d4ddf1df8, user_id=46c8c9c636d84b5cbde3151f3733ee75, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:46.321192 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67eebc-ee16-4144-bc5e-54567c237c67 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:46.331460 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1399/2807] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:45:46 2022] PUT /identity/v3/projects/1cf755f2acd146ef9f21fc1d4ddf1df8/users/46c8c9c636d84b5cbde3151f3733ee75/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:46.341877 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c0c76-f47b-468b-ac4f-65dd105a8854 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:46.342159 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c0c76-f47b-468b-ac4f-65dd105a8854 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:46.342432 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c0c76-f47b-468b-ac4f-65dd105a8854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:46.370380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-574c0c76-f47b-468b-ac4f-65dd105a8854 None None] MFA Rules not processed for user `46c8c9c636d84b5cbde3151f3733ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:46.371514 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-574c0c76-f47b-468b-ac4f-65dd105a8854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:46.438469 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1409/2808] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:46.450485 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67eebc-ee16-4144-bc5e-54567c237c67 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:46.452251 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-9c67eebc-ee16-4144-bc5e-54567c237c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:46.513115 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67eebc-ee16-4144-bc5e-54567c237c67 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:46.517001 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae67411-e962-4a3d-ac23-2dc3dbffc21d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:46.517695 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae67411-e962-4a3d-ac23-2dc3dbffc21d service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:46.517997 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae67411-e962-4a3d-ac23-2dc3dbffc21d service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:46.518289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae67411-e962-4a3d-ac23-2dc3dbffc21d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:46.576068 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae67411-e962-4a3d-ac23-2dc3dbffc21d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:46.577546 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae67411-e962-4a3d-ac23-2dc3dbffc21d service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:46.586947 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1400/2809] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:45:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:45:49.993790 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c0c76-f47b-468b-ac4f-65dd105a8854 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:50.056417 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c0c76-f47b-468b-ac4f-65dd105a8854 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:50.060949 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9cbaa0-34e8-434c-a47c-1275356d1343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:50.061598 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cbaa0-34e8-434c-a47c-1275356d1343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:50.061881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cbaa0-34e8-434c-a47c-1275356d1343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:50.062157 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cbaa0-34e8-434c-a47c-1275356d1343 admin admin] PATH_INFO: `/v3/users/d10406d49fe149b7a486afa5d9627a27` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:50.063266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9cbaa0-34e8-434c-a47c-1275356d1343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d10406d49fe149b7a486afa5d9627a27)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:50.065240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9cbaa0-34e8-434c-a47c-1275356d1343 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:50.158461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:50.292340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ae9cbaa0-34e8-434c-a47c-1275356d1343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd10406d49fe149b7a486afa5d9627a27'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:50.363200 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ae9cbaa0-34e8-434c-a47c-1275356d1343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd10406d49fe149b7a486afa5d9627a27'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:50.363907 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ae9cbaa0-34e8-434c-a47c-1275356d1343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd10406d49fe149b7a486afa5d9627a27'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:50.364613 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ae9cbaa0-34e8-434c-a47c-1275356d1343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd10406d49fe149b7a486afa5d9627a27'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:50.373022 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-ae9cbaa0-34e8-434c-a47c-1275356d1343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd10406d49fe149b7a486afa5d9627a27'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:50.391576 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1410/2810] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:45:49 2022] DELETE /identity/v3/users/d10406d49fe149b7a486afa5d9627a27 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:50.396829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a59fb2-a7eb-4519-b75c-ba082f95a760 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:50.397391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a59fb2-a7eb-4519-b75c-ba082f95a760 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:50.397508 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a59fb2-a7eb-4519-b75c-ba082f95a760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:50.416960 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-75a59fb2-a7eb-4519-b75c-ba082f95a760 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:50.418109 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-75a59fb2-a7eb-4519-b75c-ba082f95a760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:50.476482 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1401/2811] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:45:50.486153 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9cbaa0-34e8-434c-a47c-1275356d1343 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:50.488403 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-ae9cbaa0-34e8-434c-a47c-1275356d1343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:50.551694 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9cbaa0-34e8-434c-a47c-1275356d1343 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:50.555607 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53791cd-d9bb-4fb0-8d76-f47d69d2a591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:50.556258 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53791cd-d9bb-4fb0-8d76-f47d69d2a591 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:50.556588 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53791cd-d9bb-4fb0-8d76-f47d69d2a591 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:50.556865 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53791cd-d9bb-4fb0-8d76-f47d69d2a591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:50.566838 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53791cd-d9bb-4fb0-8d76-f47d69d2a591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:50.568441 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53791cd-d9bb-4fb0-8d76-f47d69d2a591 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:50.578871 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1411/2812] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:45:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:45:51.036813 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a59fb2-a7eb-4519-b75c-ba082f95a760 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:51.036813 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-75a59fb2-a7eb-4519-b75c-ba082f95a760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:51.050629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a59fb2-a7eb-4519-b75c-ba082f95a760 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:51.067692 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fa928-6c06-4b59-a2ee-9bef76f577b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:51.067928 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fa928-6c06-4b59-a2ee-9bef76f577b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:51.068278 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fa928-6c06-4b59-a2ee-9bef76f577b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:51.068473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fa928-6c06-4b59-a2ee-9bef76f577b5 admin admin] PATH_INFO: `/v3/projects/5ed5f8c3f795487dacc9b2c005118d84` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:51.069722 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0fa928-6c06-4b59-a2ee-9bef76f577b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ed5f8c3f795487dacc9b2c005118d84)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:51.071830 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0fa928-6c06-4b59-a2ee-9bef76f577b5 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:51.082930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:51.244457 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bc0fa928-6c06-4b59-a2ee-9bef76f577b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ed5f8c3f795487dacc9b2c005118d84', 'request_id': 'req-bc0fa928-6c06-4b59-a2ee-9bef76f577b5'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:51.250441 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bc0fa928-6c06-4b59-a2ee-9bef76f577b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ed5f8c3f795487dacc9b2c005118d84', 'request_id': 'req-bc0fa928-6c06-4b59-a2ee-9bef76f577b5'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:51.258223 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bc0fa928-6c06-4b59-a2ee-9bef76f577b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ed5f8c3f795487dacc9b2c005118d84', 'request_id': 'req-bc0fa928-6c06-4b59-a2ee-9bef76f577b5'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:51.328871 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bc0fa928-6c06-4b59-a2ee-9bef76f577b5 admin admin] The token cache is being invalidate because project 5ed5f8c3f795487dacc9b2c005118d84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:45:51.329195 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bc0fa928-6c06-4b59-a2ee-9bef76f577b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:51.329864 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-bc0fa928-6c06-4b59-a2ee-9bef76f577b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:51.332070 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1402/2813] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:45:51 2022] DELETE /identity/v3/projects/5ed5f8c3f795487dacc9b2c005118d84 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:51.336194 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53791cd-d9bb-4fb0-8d76-f47d69d2a591 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:51.345845 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53791cd-d9bb-4fb0-8d76-f47d69d2a591 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:51.401962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456ff38-7428-462c-9b86-4f210f579765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:51.402632 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456ff38-7428-462c-9b86-4f210f579765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:51.402914 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456ff38-7428-462c-9b86-4f210f579765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:51.403203 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456ff38-7428-462c-9b86-4f210f579765 admin admin] PATH_INFO: `/v3/users/46c8c9c636d84b5cbde3151f3733ee75` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:51.404400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456ff38-7428-462c-9b86-4f210f579765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46c8c9c636d84b5cbde3151f3733ee75)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:51.406514 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456ff38-7428-462c-9b86-4f210f579765 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:51.566426 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2456ff38-7428-462c-9b86-4f210f579765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46c8c9c636d84b5cbde3151f3733ee75'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:51.568061 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:51.631774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2456ff38-7428-462c-9b86-4f210f579765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46c8c9c636d84b5cbde3151f3733ee75'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:51.632438 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2456ff38-7428-462c-9b86-4f210f579765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46c8c9c636d84b5cbde3151f3733ee75'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:51.633051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2456ff38-7428-462c-9b86-4f210f579765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46c8c9c636d84b5cbde3151f3733ee75'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:51.640691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-2456ff38-7428-462c-9b86-4f210f579765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46c8c9c636d84b5cbde3151f3733ee75'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:51.658993 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1412/2814] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:45:51 2022] DELETE /identity/v3/users/46c8c9c636d84b5cbde3151f3733ee75 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:45:52.039579 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fa928-6c06-4b59-a2ee-9bef76f577b5 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:52.040358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-bc0fa928-6c06-4b59-a2ee-9bef76f577b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:52.057342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fa928-6c06-4b59-a2ee-9bef76f577b5 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:52.120737 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9849b4-c325-4922-b1f3-1b1ba0230f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:52.121401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9849b4-c325-4922-b1f3-1b1ba0230f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:52.121679 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9849b4-c325-4922-b1f3-1b1ba0230f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:52.122022 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9849b4-c325-4922-b1f3-1b1ba0230f39 admin admin] PATH_INFO: `/v3/projects/1cf755f2acd146ef9f21fc1d4ddf1df8` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:52.123330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9849b4-c325-4922-b1f3-1b1ba0230f39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cf755f2acd146ef9f21fc1d4ddf1df8)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:52.125522 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9849b4-c325-4922-b1f3-1b1ba0230f39 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:52.263068 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ba9849b4-c325-4922-b1f3-1b1ba0230f39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cf755f2acd146ef9f21fc1d4ddf1df8', 'request_id': 'req-ba9849b4-c325-4922-b1f3-1b1ba0230f39'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:52.269782 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ba9849b4-c325-4922-b1f3-1b1ba0230f39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cf755f2acd146ef9f21fc1d4ddf1df8', 'request_id': 'req-ba9849b4-c325-4922-b1f3-1b1ba0230f39'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:52.276529 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ba9849b4-c325-4922-b1f3-1b1ba0230f39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cf755f2acd146ef9f21fc1d4ddf1df8', 'request_id': 'req-ba9849b4-c325-4922-b1f3-1b1ba0230f39'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:52.380477 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ba9849b4-c325-4922-b1f3-1b1ba0230f39 admin admin] The token cache is being invalidate because project 1cf755f2acd146ef9f21fc1d4ddf1df8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:45:52.380705 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ba9849b4-c325-4922-b1f3-1b1ba0230f39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:52.381357 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-ba9849b4-c325-4922-b1f3-1b1ba0230f39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:45:52.382909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1403/2815] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:45:52 2022] DELETE /identity/v3/projects/1cf755f2acd146ef9f21fc1d4ddf1df8 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:45:52.388973 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe4951-54bb-4fa9-8626-068fb7339b94 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:52.389506 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe4951-54bb-4fa9-8626-068fb7339b94 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:52.389633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe4951-54bb-4fa9-8626-068fb7339b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:52.410461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-e6fe4951-54bb-4fa9-8626-068fb7339b94 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:52.411663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e6fe4951-54bb-4fa9-8626-068fb7339b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:52.474054 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1413/2816] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:45:52.482794 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f88ab4-896a-441f-988a-dae9a29789a2 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:52.483080 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f88ab4-896a-441f-988a-dae9a29789a2 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:52.483362 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f88ab4-896a-441f-988a-dae9a29789a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:52.503439 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-b0f88ab4-896a-441f-988a-dae9a29789a2 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:52.504642 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b0f88ab4-896a-441f-988a-dae9a29789a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:52.512546 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1404/2817] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:52.516795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe4951-54bb-4fa9-8626-068fb7339b94 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:52.528112 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe4951-54bb-4fa9-8626-068fb7339b94 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:52.532626 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808aeb3-ed7c-4c08-b62e-c771436803f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:52.533245 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808aeb3-ed7c-4c08-b62e-c771436803f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:52.533546 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808aeb3-ed7c-4c08-b62e-c771436803f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:52.533840 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808aeb3-ed7c-4c08-b62e-c771436803f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:52.535085 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808aeb3-ed7c-4c08-b62e-c771436803f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:45:52.535382 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808aeb3-ed7c-4c08-b62e-c771436803f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:52.536952 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808aeb3-ed7c-4c08-b62e-c771436803f8 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:52.548703 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1414/2818] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 10:45:52.553019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f88ab4-896a-441f-988a-dae9a29789a2 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:52.563533 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f88ab4-896a-441f-988a-dae9a29789a2 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:52.573590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97a75c-f876-46b3-8220-9f8f106ae44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:52.574345 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97a75c-f876-46b3-8220-9f8f106ae44a admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:52.574671 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97a75c-f876-46b3-8220-9f8f106ae44a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:52.574984 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97a75c-f876-46b3-8220-9f8f106ae44a admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:52.575775 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97a75c-f876-46b3-8220-9f8f106ae44a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:52.578049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97a75c-f876-46b3-8220-9f8f106ae44a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:52.678173 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:52.687548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1405/2819] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:45:52 2022] POST /identity/v3/projects => generated 391 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:45:52.688679 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808aeb3-ed7c-4c08-b62e-c771436803f8 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:52.699356 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808aeb3-ed7c-4c08-b62e-c771436803f8 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:52.755616 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36f8c5-4b85-486a-ab4b-a38912c233cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:52.756216 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36f8c5-4b85-486a-ab4b-a38912c233cc admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:52.756496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36f8c5-4b85-486a-ab4b-a38912c233cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:52.756767 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36f8c5-4b85-486a-ab4b-a38912c233cc admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:52.757429 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36f8c5-4b85-486a-ab4b-a38912c233cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:52.759373 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36f8c5-4b85-486a-ab4b-a38912c233cc admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:52.850815 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1415/2820] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:45:52 2022] POST /identity/v3/users => generated 296 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:45:52.854900 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97a75c-f876-46b3-8220-9f8f106ae44a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:52.866001 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97a75c-f876-46b3-8220-9f8f106ae44a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:52.871019 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8142f7-6113-41b5-a7b7-d7ac07b3f792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:52.871606 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8142f7-6113-41b5-a7b7-d7ac07b3f792 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:52.871882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8142f7-6113-41b5-a7b7-d7ac07b3f792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:52.872154 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8142f7-6113-41b5-a7b7-d7ac07b3f792 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:52.872783 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8142f7-6113-41b5-a7b7-d7ac07b3f792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:52.874400 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8142f7-6113-41b5-a7b7-d7ac07b3f792 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:52.884160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1406/2821] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:52.888105 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36f8c5-4b85-486a-ab4b-a38912c233cc admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:52.898467 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36f8c5-4b85-486a-ab4b-a38912c233cc admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:52.902974 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d670c-364f-4180-92c8-8a3508bda130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:52.903556 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d670c-364f-4180-92c8-8a3508bda130 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:52.903853 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d670c-364f-4180-92c8-8a3508bda130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:52.904124 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d670c-364f-4180-92c8-8a3508bda130 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:52.904770 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d670c-364f-4180-92c8-8a3508bda130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:52.906385 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d670c-364f-4180-92c8-8a3508bda130 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:52.917551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1416/2822] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:45:52.929104 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8142f7-6113-41b5-a7b7-d7ac07b3f792 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:52.941784 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8142f7-6113-41b5-a7b7-d7ac07b3f792 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:52.946667 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d446039-51a4-4984-af43-7323ad5980b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:52.947332 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d446039-51a4-4984-af43-7323ad5980b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:52.947588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d446039-51a4-4984-af43-7323ad5980b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:52.947897 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d446039-51a4-4984-af43-7323ad5980b2 admin admin] PATH_INFO: `/v3/projects/4db8530cdc6c47b78fc684d4e0389e26/users/00554da12d7f4b4a8abce4e0d9f0a660/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:52.962930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d446039-51a4-4984-af43-7323ad5980b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4db8530cdc6c47b78fc684d4e0389e26, user_id=00554da12d7f4b4a8abce4e0d9f0a660, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:52.965653 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d446039-51a4-4984-af43-7323ad5980b2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:53.027205 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1407/2823] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:45:52 2022] PUT /identity/v3/projects/4db8530cdc6c47b78fc684d4e0389e26/users/00554da12d7f4b4a8abce4e0d9f0a660/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:45:53.038181 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c46b6-c049-43e0-b84a-4eaf91f9919b None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:53.038463 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c46b6-c049-43e0-b84a-4eaf91f9919b None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:53.038738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c46b6-c049-43e0-b84a-4eaf91f9919b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:53.067333 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-951c46b6-c049-43e0-b84a-4eaf91f9919b None None] MFA Rules not processed for user `00554da12d7f4b4a8abce4e0d9f0a660`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:45:53.068568 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-951c46b6-c049-43e0-b84a-4eaf91f9919b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:53.087208 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:45:53.132427 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1417/2824] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:45:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:45:53.143685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d446039-51a4-4984-af43-7323ad5980b2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:45:53.145868 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6d446039-51a4-4984-af43-7323ad5980b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:45:53.202299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d446039-51a4-4984-af43-7323ad5980b2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:45:53.205951 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af719a-c92f-476d-abd2-671c23b53370 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:45:53.206581 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af719a-c92f-476d-abd2-671c23b53370 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:45:53.206865 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af719a-c92f-476d-abd2-671c23b53370 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:45:53.207143 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af719a-c92f-476d-abd2-671c23b53370 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:45:53.215336 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af719a-c92f-476d-abd2-671c23b53370 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:45:53.216750 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af719a-c92f-476d-abd2-671c23b53370 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:45:53.225492 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1408/2825] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:45:53 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:46:03.168995 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=18 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:46:03.221452 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:46:12.492020 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c46b6-c049-43e0-b84a-4eaf91f9919b None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:12.493528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-951c46b6-c049-43e0-b84a-4eaf91f9919b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:12.559761 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c46b6-c049-43e0-b84a-4eaf91f9919b None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:12.564571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feab7bc-a7cc-4205-aabd-5bf14a402b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:12.565233 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feab7bc-a7cc-4205-aabd-5bf14a402b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:12.565518 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feab7bc-a7cc-4205-aabd-5bf14a402b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:12.565795 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feab7bc-a7cc-4205-aabd-5bf14a402b7d admin admin] PATH_INFO: `/v3/users/e264aeff7a904b03bf3116c5b52620b0` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:12.566914 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feab7bc-a7cc-4205-aabd-5bf14a402b7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e264aeff7a904b03bf3116c5b52620b0)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:12.569449 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feab7bc-a7cc-4205-aabd-5bf14a402b7d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:12.765893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-9feab7bc-a7cc-4205-aabd-5bf14a402b7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e264aeff7a904b03bf3116c5b52620b0'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:12.831614 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-9feab7bc-a7cc-4205-aabd-5bf14a402b7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e264aeff7a904b03bf3116c5b52620b0'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:12.832237 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-9feab7bc-a7cc-4205-aabd-5bf14a402b7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e264aeff7a904b03bf3116c5b52620b0'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:12.832893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-9feab7bc-a7cc-4205-aabd-5bf14a402b7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e264aeff7a904b03bf3116c5b52620b0'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:12.841003 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-9feab7bc-a7cc-4205-aabd-5bf14a402b7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e264aeff7a904b03bf3116c5b52620b0'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:12.858263 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1418/2826] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:46:12 2022] DELETE /identity/v3/users/e264aeff7a904b03bf3116c5b52620b0 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:46:12.862882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b059b89-fb7f-4bf9-98f9-ea5932945090 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:12.863367 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b059b89-fb7f-4bf9-98f9-ea5932945090 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:12.863473 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b059b89-fb7f-4bf9-98f9-ea5932945090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:12.883196 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1b059b89-fb7f-4bf9-98f9-ea5932945090 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:46:12.884372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1b059b89-fb7f-4bf9-98f9-ea5932945090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:12.940117 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1409/2827] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:46:12.950446 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feab7bc-a7cc-4205-aabd-5bf14a402b7d admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:12.951819 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-9feab7bc-a7cc-4205-aabd-5bf14a402b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:13.017853 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feab7bc-a7cc-4205-aabd-5bf14a402b7d admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:13.022053 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c2b33-a7bd-4c7e-bd1d-ba0d8d74cab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:13.022662 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c2b33-a7bd-4c7e-bd1d-ba0d8d74cab5 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:13.023187 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c2b33-a7bd-4c7e-bd1d-ba0d8d74cab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:13.023456 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c2b33-a7bd-4c7e-bd1d-ba0d8d74cab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:13.033234 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c2b33-a7bd-4c7e-bd1d-ba0d8d74cab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:13.034720 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c2b33-a7bd-4c7e-bd1d-ba0d8d74cab5 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:13.044718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1419/2828] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:46:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:46:13.501931 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b059b89-fb7f-4bf9-98f9-ea5932945090 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:13.503358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1b059b89-fb7f-4bf9-98f9-ea5932945090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:13.514991 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b059b89-fb7f-4bf9-98f9-ea5932945090 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:13.520060 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179a383-d99f-49d8-843c-d412ccea4a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:13.520707 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179a383-d99f-49d8-843c-d412ccea4a01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:13.521016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179a383-d99f-49d8-843c-d412ccea4a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:13.521498 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179a383-d99f-49d8-843c-d412ccea4a01 admin admin] PATH_INFO: `/v3/projects/e12d80e5675745379eba8b615fb61048` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:13.522634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179a383-d99f-49d8-843c-d412ccea4a01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e12d80e5675745379eba8b615fb61048)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:13.525467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179a383-d99f-49d8-843c-d412ccea4a01 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:13.631859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6179a383-d99f-49d8-843c-d412ccea4a01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e12d80e5675745379eba8b615fb61048', 'request_id': 'req-6179a383-d99f-49d8-843c-d412ccea4a01'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:13.637456 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6179a383-d99f-49d8-843c-d412ccea4a01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e12d80e5675745379eba8b615fb61048', 'request_id': 'req-6179a383-d99f-49d8-843c-d412ccea4a01'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:13.643528 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6179a383-d99f-49d8-843c-d412ccea4a01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e12d80e5675745379eba8b615fb61048', 'request_id': 'req-6179a383-d99f-49d8-843c-d412ccea4a01'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:13.688282 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6179a383-d99f-49d8-843c-d412ccea4a01 admin admin] The token cache is being invalidate because project e12d80e5675745379eba8b615fb61048 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:46:13.688576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6179a383-d99f-49d8-843c-d412ccea4a01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:13.689245 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6179a383-d99f-49d8-843c-d412ccea4a01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:13.690738 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1410/2829] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:46:13 2022] DELETE /identity/v3/projects/e12d80e5675745379eba8b615fb61048 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:46:13.695656 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b31c40-e599-4af2-bb80-23564959a1a4 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:13.700647 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b31c40-e599-4af2-bb80-23564959a1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:13.700647 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b31c40-e599-4af2-bb80-23564959a1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:13.716137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-87b31c40-e599-4af2-bb80-23564959a1a4 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:46:13.717356 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-87b31c40-e599-4af2-bb80-23564959a1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:13.733904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:46:13.774176 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1420/2830] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:46:13.781412 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955b9e-99f5-42b7-a2c2-7c9b9d9cb66c None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:13.781707 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955b9e-99f5-42b7-a2c2-7c9b9d9cb66c None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:13.781988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955b9e-99f5-42b7-a2c2-7c9b9d9cb66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:13.802568 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-89955b9e-99f5-42b7-a2c2-7c9b9d9cb66c None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:46:13.803815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-89955b9e-99f5-42b7-a2c2-7c9b9d9cb66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:13.811839 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1411/2831] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:46:13.816213 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b31c40-e599-4af2-bb80-23564959a1a4 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:13.826327 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b31c40-e599-4af2-bb80-23564959a1a4 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:13.831093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0959730-dc87-4a88-a8b1-70ef974a28f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:13.831687 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0959730-dc87-4a88-a8b1-70ef974a28f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:13.831965 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0959730-dc87-4a88-a8b1-70ef974a28f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:13.832236 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0959730-dc87-4a88-a8b1-70ef974a28f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:13.832928 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0959730-dc87-4a88-a8b1-70ef974a28f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:46:13.833255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0959730-dc87-4a88-a8b1-70ef974a28f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:13.835338 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0959730-dc87-4a88-a8b1-70ef974a28f4 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:13.846841 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1421/2832] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:46:13.852143 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955b9e-99f5-42b7-a2c2-7c9b9d9cb66c None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:13.863144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955b9e-99f5-42b7-a2c2-7c9b9d9cb66c None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:13.868187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65ec50-9379-4183-890f-377568ef779c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:13.868765 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65ec50-9379-4183-890f-377568ef779c admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:13.869049 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65ec50-9379-4183-890f-377568ef779c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:13.869360 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65ec50-9379-4183-890f-377568ef779c admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:13.870100 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65ec50-9379-4183-890f-377568ef779c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:13.872016 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65ec50-9379-4183-890f-377568ef779c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:13.992875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1412/2833] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:46:13 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:46:13.997118 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0959730-dc87-4a88-a8b1-70ef974a28f4 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:14.007558 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0959730-dc87-4a88-a8b1-70ef974a28f4 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:14.062424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c526b4c-d532-4784-a4c3-dc66f72fceae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:14.062953 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c526b4c-d532-4784-a4c3-dc66f72fceae admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:14.063234 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c526b4c-d532-4784-a4c3-dc66f72fceae admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:14.063510 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c526b4c-d532-4784-a4c3-dc66f72fceae admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:14.064148 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c526b4c-d532-4784-a4c3-dc66f72fceae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:14.066170 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c526b4c-d532-4784-a4c3-dc66f72fceae admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:14.134930 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1422/2834] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:46:13 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:46:14.139101 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65ec50-9379-4183-890f-377568ef779c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:14.150551 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65ec50-9379-4183-890f-377568ef779c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:14.155319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57b76f-e575-4646-9580-760ad8f73059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:14.155924 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57b76f-e575-4646-9580-760ad8f73059 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:14.156362 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57b76f-e575-4646-9580-760ad8f73059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:14.156644 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57b76f-e575-4646-9580-760ad8f73059 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:14.157342 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57b76f-e575-4646-9580-760ad8f73059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:14.158958 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57b76f-e575-4646-9580-760ad8f73059 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:14.168897 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1413/2835] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:46:14.172742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c526b4c-d532-4784-a4c3-dc66f72fceae admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:14.182768 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c526b4c-d532-4784-a4c3-dc66f72fceae admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:14.187357 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f993a-0699-4599-9254-782fab5c9021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:14.187953 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f993a-0699-4599-9254-782fab5c9021 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:14.188235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f993a-0699-4599-9254-782fab5c9021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:14.188516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f993a-0699-4599-9254-782fab5c9021 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:14.189192 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f993a-0699-4599-9254-782fab5c9021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:14.190802 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f993a-0699-4599-9254-782fab5c9021 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:14.202633 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1423/2836] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:46:14.206753 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57b76f-e575-4646-9580-760ad8f73059 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:14.217472 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57b76f-e575-4646-9580-760ad8f73059 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:14.222413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f900c14-6f1c-4415-a4ed-111f1c3e2f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:14.223198 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f900c14-6f1c-4415-a4ed-111f1c3e2f48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:14.223546 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f900c14-6f1c-4415-a4ed-111f1c3e2f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:14.223892 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f900c14-6f1c-4415-a4ed-111f1c3e2f48 admin admin] PATH_INFO: `/v3/projects/5523ef2925484083b51d4cc2eb6b815e/users/9e98bb542bf94369b03df77dfcccd578/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:14.239093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f900c14-6f1c-4415-a4ed-111f1c3e2f48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5523ef2925484083b51d4cc2eb6b815e, user_id=9e98bb542bf94369b03df77dfcccd578, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:14.241756 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f900c14-6f1c-4415-a4ed-111f1c3e2f48 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:14.398326 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1414/2837] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:46:14 2022] PUT /identity/v3/projects/5523ef2925484083b51d4cc2eb6b815e/users/9e98bb542bf94369b03df77dfcccd578/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:46:14.409996 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518a8c7-9c01-4834-a566-93e04c4b5f98 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:14.410282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518a8c7-9c01-4834-a566-93e04c4b5f98 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:14.410557 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518a8c7-9c01-4834-a566-93e04c4b5f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:14.438247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-4518a8c7-9c01-4834-a566-93e04c4b5f98 None None] MFA Rules not processed for user `9e98bb542bf94369b03df77dfcccd578`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:46:14.439476 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4518a8c7-9c01-4834-a566-93e04c4b5f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:14.504239 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1424/2838] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:46:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:46:14.516147 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f900c14-6f1c-4415-a4ed-111f1c3e2f48 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:14.517858 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-6f900c14-6f1c-4415-a4ed-111f1c3e2f48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:14.539715 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:46:14.577692 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f900c14-6f1c-4415-a4ed-111f1c3e2f48 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:14.581536 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f0b63-e35e-4546-80ba-85b8d3f48619 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:14.582167 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f0b63-e35e-4546-80ba-85b8d3f48619 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:14.589971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f0b63-e35e-4546-80ba-85b8d3f48619 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:14.589971 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f0b63-e35e-4546-80ba-85b8d3f48619 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:14.591001 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f0b63-e35e-4546-80ba-85b8d3f48619 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:14.592429 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f0b63-e35e-4546-80ba-85b8d3f48619 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:14.601193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1415/2839] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:46:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:46:14.627241 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518a8c7-9c01-4834-a566-93e04c4b5f98 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:14.647015 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:46:14.692887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518a8c7-9c01-4834-a566-93e04c4b5f98 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:14.698218 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39261a8d-4041-4ec5-9507-21f58ed0ad94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:14.698849 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39261a8d-4041-4ec5-9507-21f58ed0ad94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:14.699150 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39261a8d-4041-4ec5-9507-21f58ed0ad94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:14.699408 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39261a8d-4041-4ec5-9507-21f58ed0ad94 admin admin] PATH_INFO: `/v3/users/9e98bb542bf94369b03df77dfcccd578` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:14.700543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39261a8d-4041-4ec5-9507-21f58ed0ad94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e98bb542bf94369b03df77dfcccd578)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:14.702485 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39261a8d-4041-4ec5-9507-21f58ed0ad94 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:15.057329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-39261a8d-4041-4ec5-9507-21f58ed0ad94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e98bb542bf94369b03df77dfcccd578'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:15.197287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-39261a8d-4041-4ec5-9507-21f58ed0ad94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e98bb542bf94369b03df77dfcccd578'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:15.197897 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-39261a8d-4041-4ec5-9507-21f58ed0ad94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e98bb542bf94369b03df77dfcccd578'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:15.198505 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-39261a8d-4041-4ec5-9507-21f58ed0ad94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e98bb542bf94369b03df77dfcccd578'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:15.206016 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-39261a8d-4041-4ec5-9507-21f58ed0ad94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e98bb542bf94369b03df77dfcccd578'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:15.224299 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1425/2840] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:46:14 2022] DELETE /identity/v3/users/9e98bb542bf94369b03df77dfcccd578 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:46:15.229581 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9b60d-f9e2-4648-954b-6c74352e7109 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:15.230086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9b60d-f9e2-4648-954b-6c74352e7109 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:15.230199 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9b60d-f9e2-4648-954b-6c74352e7109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:15.248860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-fed9b60d-f9e2-4648-954b-6c74352e7109 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:46:15.250061 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-fed9b60d-f9e2-4648-954b-6c74352e7109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:15.308766 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1416/2841] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:46:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:46:15.318461 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39261a8d-4041-4ec5-9507-21f58ed0ad94 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:15.319885 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-39261a8d-4041-4ec5-9507-21f58ed0ad94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:15.377410 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39261a8d-4041-4ec5-9507-21f58ed0ad94 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:15.381292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6367f69-8520-4fff-83ff-5c114b4249d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:15.381992 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6367f69-8520-4fff-83ff-5c114b4249d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:15.382270 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6367f69-8520-4fff-83ff-5c114b4249d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:15.382544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6367f69-8520-4fff-83ff-5c114b4249d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:15.391691 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6367f69-8520-4fff-83ff-5c114b4249d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:15.396016 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6367f69-8520-4fff-83ff-5c114b4249d3 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:15.403329 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1426/2842] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:46:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:46:15.711274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9b60d-f9e2-4648-954b-6c74352e7109 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:15.712725 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-fed9b60d-f9e2-4648-954b-6c74352e7109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:15.724538 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9b60d-f9e2-4648-954b-6c74352e7109 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:15.729204 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59cd866-c7e7-433d-a3fb-1945cab49800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:15.729824 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cd866-c7e7-433d-a3fb-1945cab49800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:15.730100 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cd866-c7e7-433d-a3fb-1945cab49800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:15.730372 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cd866-c7e7-433d-a3fb-1945cab49800 admin admin] PATH_INFO: `/v3/users/00554da12d7f4b4a8abce4e0d9f0a660` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:15.731477 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59cd866-c7e7-433d-a3fb-1945cab49800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00554da12d7f4b4a8abce4e0d9f0a660)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:15.733400 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59cd866-c7e7-433d-a3fb-1945cab49800 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:15.839341 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6367f69-8520-4fff-83ff-5c114b4249d3 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:15.842740 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c6367f69-8520-4fff-83ff-5c114b4249d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:15.853743 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6367f69-8520-4fff-83ff-5c114b4249d3 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:15.858328 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2218e-9fa5-4aab-91a6-07daefb85871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:15.858958 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2218e-9fa5-4aab-91a6-07daefb85871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:15.859241 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2218e-9fa5-4aab-91a6-07daefb85871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:15.859514 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2218e-9fa5-4aab-91a6-07daefb85871 admin admin] PATH_INFO: `/v3/projects/5523ef2925484083b51d4cc2eb6b815e` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:15.860649 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2218e-9fa5-4aab-91a6-07daefb85871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5523ef2925484083b51d4cc2eb6b815e)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:15.866715 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2218e-9fa5-4aab-91a6-07daefb85871 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:15.903268 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e59cd866-c7e7-433d-a3fb-1945cab49800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00554da12d7f4b4a8abce4e0d9f0a660'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:15.903914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e59cd866-c7e7-433d-a3fb-1945cab49800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00554da12d7f4b4a8abce4e0d9f0a660'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:15.988750 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:46:15.998641 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-31a2218e-9fa5-4aab-91a6-07daefb85871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5523ef2925484083b51d4cc2eb6b815e', 'request_id': 'req-31a2218e-9fa5-4aab-91a6-07daefb85871'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:16.003122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e59cd866-c7e7-433d-a3fb-1945cab49800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00554da12d7f4b4a8abce4e0d9f0a660'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:16.003752 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e59cd866-c7e7-433d-a3fb-1945cab49800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00554da12d7f4b4a8abce4e0d9f0a660'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:16.004380 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-31a2218e-9fa5-4aab-91a6-07daefb85871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5523ef2925484083b51d4cc2eb6b815e', 'request_id': 'req-31a2218e-9fa5-4aab-91a6-07daefb85871'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:16.011206 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-31a2218e-9fa5-4aab-91a6-07daefb85871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5523ef2925484083b51d4cc2eb6b815e', 'request_id': 'req-31a2218e-9fa5-4aab-91a6-07daefb85871'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:16.017290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-e59cd866-c7e7-433d-a3fb-1945cab49800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00554da12d7f4b4a8abce4e0d9f0a660'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:16.029819 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1417/2843] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:46:15 2022] DELETE /identity/v3/users/00554da12d7f4b4a8abce4e0d9f0a660 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:46:16.034145 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92b77c-a35b-4239-8038-b3749bf247f8 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:16.034432 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92b77c-a35b-4239-8038-b3749bf247f8 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:16.034709 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92b77c-a35b-4239-8038-b3749bf247f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:16.052964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-ec92b77c-a35b-4239-8038-b3749bf247f8 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:46:16.054708 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ec92b77c-a35b-4239-8038-b3749bf247f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:16.084208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:46:16.094099 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-31a2218e-9fa5-4aab-91a6-07daefb85871 admin admin] The token cache is being invalidate because project 5523ef2925484083b51d4cc2eb6b815e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:46:16.094396 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-31a2218e-9fa5-4aab-91a6-07daefb85871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:16.095045 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-31a2218e-9fa5-4aab-91a6-07daefb85871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:16.096548 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1427/2844] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:46:15 2022] DELETE /identity/v3/projects/5523ef2925484083b51d4cc2eb6b815e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:46:16.125944 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1418/2845] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:46:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:46:16.133779 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2218e-9fa5-4aab-91a6-07daefb85871 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:16.135221 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-31a2218e-9fa5-4aab-91a6-07daefb85871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:16.200196 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2218e-9fa5-4aab-91a6-07daefb85871 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:16.251826 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b6536-50d0-43d4-9230-a4084d2a3c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:16.252453 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b6536-50d0-43d4-9230-a4084d2a3c3b service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:16.252727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b6536-50d0-43d4-9230-a4084d2a3c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:16.253001 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b6536-50d0-43d4-9230-a4084d2a3c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:16.263056 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b6536-50d0-43d4-9230-a4084d2a3c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:16.264489 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b6536-50d0-43d4-9230-a4084d2a3c3b service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:16.275172 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1428/2846] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:46:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:46:16.684793 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92b77c-a35b-4239-8038-b3749bf247f8 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:16.686277 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-ec92b77c-a35b-4239-8038-b3749bf247f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:16.698242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92b77c-a35b-4239-8038-b3749bf247f8 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:16.702912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521c68c-68a0-4260-a6f6-c0d20d29c1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:16.703547 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521c68c-68a0-4260-a6f6-c0d20d29c1a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:16.703829 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521c68c-68a0-4260-a6f6-c0d20d29c1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:16.704111 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521c68c-68a0-4260-a6f6-c0d20d29c1a7 admin admin] PATH_INFO: `/v3/projects/4db8530cdc6c47b78fc684d4e0389e26` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:16.705275 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521c68c-68a0-4260-a6f6-c0d20d29c1a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4db8530cdc6c47b78fc684d4e0389e26)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:16.707178 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521c68c-68a0-4260-a6f6-c0d20d29c1a7 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:16.856006 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c521c68c-68a0-4260-a6f6-c0d20d29c1a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4db8530cdc6c47b78fc684d4e0389e26', 'request_id': 'req-c521c68c-68a0-4260-a6f6-c0d20d29c1a7'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:16.861531 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c521c68c-68a0-4260-a6f6-c0d20d29c1a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4db8530cdc6c47b78fc684d4e0389e26', 'request_id': 'req-c521c68c-68a0-4260-a6f6-c0d20d29c1a7'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:16.867851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c521c68c-68a0-4260-a6f6-c0d20d29c1a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4db8530cdc6c47b78fc684d4e0389e26', 'request_id': 'req-c521c68c-68a0-4260-a6f6-c0d20d29c1a7'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:16.933437 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c521c68c-68a0-4260-a6f6-c0d20d29c1a7 admin admin] The token cache is being invalidate because project 4db8530cdc6c47b78fc684d4e0389e26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:46:16.933731 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c521c68c-68a0-4260-a6f6-c0d20d29c1a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:16.934354 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-c521c68c-68a0-4260-a6f6-c0d20d29c1a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:16.935865 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1419/2847] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:46:16 2022] DELETE /identity/v3/projects/4db8530cdc6c47b78fc684d4e0389e26 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:46:16.946031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19005dc4-aa6e-43b7-889c-5da7b2da2ee7 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:16.946031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19005dc4-aa6e-43b7-889c-5da7b2da2ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:16.946031 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19005dc4-aa6e-43b7-889c-5da7b2da2ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:16.961358 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-19005dc4-aa6e-43b7-889c-5da7b2da2ee7 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:46:16.962566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-19005dc4-aa6e-43b7-889c-5da7b2da2ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:17.019950 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1429/2848] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:46:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:46:17.029567 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f9dfb-395f-4fec-a246-95f63d8d1c98 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:17.029883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f9dfb-395f-4fec-a246-95f63d8d1c98 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:17.030161 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f9dfb-395f-4fec-a246-95f63d8d1c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:17.049904 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-028f9dfb-395f-4fec-a246-95f63d8d1c98 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:46:17.051102 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-028f9dfb-395f-4fec-a246-95f63d8d1c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:17.058891 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1420/2849] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:46:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:46:17.063113 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19005dc4-aa6e-43b7-889c-5da7b2da2ee7 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:17.073103 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19005dc4-aa6e-43b7-889c-5da7b2da2ee7 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:17.077718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188fab3-f314-4855-96c4-316992dd2b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:17.078349 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188fab3-f314-4855-96c4-316992dd2b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:17.078624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188fab3-f314-4855-96c4-316992dd2b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:17.078897 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188fab3-f314-4855-96c4-316992dd2b0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:17.079544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188fab3-f314-4855-96c4-316992dd2b0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:46:17.079846 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188fab3-f314-4855-96c4-316992dd2b0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:17.081533 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188fab3-f314-4855-96c4-316992dd2b0d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:17.093528 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1430/2850] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:46:17.097781 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f9dfb-395f-4fec-a246-95f63d8d1c98 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:17.107047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f9dfb-395f-4fec-a246-95f63d8d1c98 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:17.111747 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a636b-2eae-470a-8498-2c973cdf8f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:17.112350 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a636b-2eae-470a-8498-2c973cdf8f96 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:17.112631 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a636b-2eae-470a-8498-2c973cdf8f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:17.112909 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a636b-2eae-470a-8498-2c973cdf8f96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:17.113619 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a636b-2eae-470a-8498-2c973cdf8f96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:17.115558 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a636b-2eae-470a-8498-2c973cdf8f96 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:17.200341 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1421/2851] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:46:17 2022] POST /identity/v3/projects => generated 397 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:46:17.204268 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188fab3-f314-4855-96c4-316992dd2b0d admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:17.214821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188fab3-f314-4855-96c4-316992dd2b0d admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:17.219534 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:46:17.270726 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac4bd9-3de3-464d-9b16-c38f3eb04d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:17.271322 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac4bd9-3de3-464d-9b16-c38f3eb04d88 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:17.271597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac4bd9-3de3-464d-9b16-c38f3eb04d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:17.271867 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac4bd9-3de3-464d-9b16-c38f3eb04d88 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:17.273429 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac4bd9-3de3-464d-9b16-c38f3eb04d88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:17.275319 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac4bd9-3de3-464d-9b16-c38f3eb04d88 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:17.362876 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1431/2852] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:46:17 2022] POST /identity/v3/users => generated 299 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:46:17.367541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a636b-2eae-470a-8498-2c973cdf8f96 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:17.377914 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a636b-2eae-470a-8498-2c973cdf8f96 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:17.383243 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a13c29-c1bc-47c0-af25-2a80796d901c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:17.383832 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a13c29-c1bc-47c0-af25-2a80796d901c admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:17.384113 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a13c29-c1bc-47c0-af25-2a80796d901c admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:17.384423 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a13c29-c1bc-47c0-af25-2a80796d901c admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:17.385076 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a13c29-c1bc-47c0-af25-2a80796d901c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:17.386673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a13c29-c1bc-47c0-af25-2a80796d901c admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:17.396562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1422/2853] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:46:17.400718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac4bd9-3de3-464d-9b16-c38f3eb04d88 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:17.410517 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac4bd9-3de3-464d-9b16-c38f3eb04d88 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:17.415119 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf671d5f-bdf2-40e2-8b03-0e2b43141a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:17.420893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf671d5f-bdf2-40e2-8b03-0e2b43141a58 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:17.420893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf671d5f-bdf2-40e2-8b03-0e2b43141a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:17.420893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf671d5f-bdf2-40e2-8b03-0e2b43141a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:17.420893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf671d5f-bdf2-40e2-8b03-0e2b43141a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:17.420893 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf671d5f-bdf2-40e2-8b03-0e2b43141a58 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:17.429401 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1432/2854] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:46:17.433465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a13c29-c1bc-47c0-af25-2a80796d901c admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:17.443355 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a13c29-c1bc-47c0-af25-2a80796d901c admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:17.448618 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb8741-0e3b-4d9c-97ca-62c52a388032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:17.449290 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb8741-0e3b-4d9c-97ca-62c52a388032 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:17.449574 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb8741-0e3b-4d9c-97ca-62c52a388032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:17.449879 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb8741-0e3b-4d9c-97ca-62c52a388032 admin admin] PATH_INFO: `/v3/projects/60a5ccc373a848428d3aac1632d86a93/users/5bd9b9890e6446cf956acb66be9a2c5f/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:17.464106 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb8741-0e3b-4d9c-97ca-62c52a388032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a5ccc373a848428d3aac1632d86a93, user_id=5bd9b9890e6446cf956acb66be9a2c5f, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:17.466696 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb8741-0e3b-4d9c-97ca-62c52a388032 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:17.546007 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1423/2855] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:46:17 2022] PUT /identity/v3/projects/60a5ccc373a848428d3aac1632d86a93/users/5bd9b9890e6446cf956acb66be9a2c5f/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:46:17.556877 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0815a-1bd1-46f9-a4ba-10a51eb7bf10 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:17.557188 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0815a-1bd1-46f9-a4ba-10a51eb7bf10 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:17.557467 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0815a-1bd1-46f9-a4ba-10a51eb7bf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:17.585969 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-c6f0815a-1bd1-46f9-a4ba-10a51eb7bf10 None None] MFA Rules not processed for user `5bd9b9890e6446cf956acb66be9a2c5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:46:17.587207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-c6f0815a-1bd1-46f9-a4ba-10a51eb7bf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:17.650004 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1433/2856] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:46:17 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:46:17.661289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb8741-0e3b-4d9c-97ca-62c52a388032 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:17.663688 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-0bdb8741-0e3b-4d9c-97ca-62c52a388032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:17.720225 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb8741-0e3b-4d9c-97ca-62c52a388032 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:17.724087 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faef996-c2e3-499a-a8df-e61ae1d13ee5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:17.724690 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faef996-c2e3-499a-a8df-e61ae1d13ee5 service cinder] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:17.724970 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faef996-c2e3-499a-a8df-e61ae1d13ee5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:17.725318 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faef996-c2e3-499a-a8df-e61ae1d13ee5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:17.733953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faef996-c2e3-499a-a8df-e61ae1d13ee5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:17.735365 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faef996-c2e3-499a-a8df-e61ae1d13ee5 service cinder] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:17.738773 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:46:17.744627 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1424/2857] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:46:17 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:46:26.778744 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0815a-1bd1-46f9-a4ba-10a51eb7bf10 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:26.854144 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0815a-1bd1-46f9-a4ba-10a51eb7bf10 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:26.854144 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d9140-b795-488a-b555-521b744aa92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:26.854144 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d9140-b795-488a-b555-521b744aa92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:26.854144 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d9140-b795-488a-b555-521b744aa92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:26.854668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d9140-b795-488a-b555-521b744aa92e admin admin] PATH_INFO: `/v3/users/5bd9b9890e6446cf956acb66be9a2c5f` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:26.854668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d9140-b795-488a-b555-521b744aa92e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bd9b9890e6446cf956acb66be9a2c5f)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:26.854668 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d9140-b795-488a-b555-521b744aa92e admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:26.870643 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:46:27.022172 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-909d9140-b795-488a-b555-521b744aa92e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bd9b9890e6446cf956acb66be9a2c5f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:27.134431 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-909d9140-b795-488a-b555-521b744aa92e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bd9b9890e6446cf956acb66be9a2c5f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:27.134998 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-909d9140-b795-488a-b555-521b744aa92e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bd9b9890e6446cf956acb66be9a2c5f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:27.135631 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-909d9140-b795-488a-b555-521b744aa92e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bd9b9890e6446cf956acb66be9a2c5f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:27.142927 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-909d9140-b795-488a-b555-521b744aa92e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bd9b9890e6446cf956acb66be9a2c5f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:27.160415 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1434/2858] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:46:26 2022] DELETE /identity/v3/users/5bd9b9890e6446cf956acb66be9a2c5f => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:46:27.165663 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ec378-0ee4-44fd-83de-1e1e061c9008 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:27.166145 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ec378-0ee4-44fd-83de-1e1e061c9008 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:27.166253 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ec378-0ee4-44fd-83de-1e1e061c9008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:27.184714 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-864ec378-0ee4-44fd-83de-1e1e061c9008 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:46:27.185953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-864ec378-0ee4-44fd-83de-1e1e061c9008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:27.251046 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1425/2859] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:46:27.260887 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d9140-b795-488a-b555-521b744aa92e admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:27.262348 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-909d9140-b795-488a-b555-521b744aa92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:27.323911 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d9140-b795-488a-b555-521b744aa92e admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:27.328160 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c758ab-3507-4794-bbed-05c3e95f4449 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:27.328763 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c758ab-3507-4794-bbed-05c3e95f4449 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:27.329047 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c758ab-3507-4794-bbed-05c3e95f4449 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:27.329353 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c758ab-3507-4794-bbed-05c3e95f4449 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:27.338798 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c758ab-3507-4794-bbed-05c3e95f4449 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:27.340229 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c758ab-3507-4794-bbed-05c3e95f4449 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:27.349665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1435/2860] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:46:27.786590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ec378-0ee4-44fd-83de-1e1e061c9008 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:27.788037 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-864ec378-0ee4-44fd-83de-1e1e061c9008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:27.800274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ec378-0ee4-44fd-83de-1e1e061c9008 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:27.805615 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2e3b3-0030-4147-82df-6dc157679325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:27.806054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2e3b3-0030-4147-82df-6dc157679325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:27.806333 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2e3b3-0030-4147-82df-6dc157679325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:27.806609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2e3b3-0030-4147-82df-6dc157679325 admin admin] PATH_INFO: `/v3/projects/60a5ccc373a848428d3aac1632d86a93` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:27.807741 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2e3b3-0030-4147-82df-6dc157679325 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60a5ccc373a848428d3aac1632d86a93)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:27.813139 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2e3b3-0030-4147-82df-6dc157679325 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:27.888581 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-81f2e3b3-0030-4147-82df-6dc157679325 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60a5ccc373a848428d3aac1632d86a93', 'request_id': 'req-81f2e3b3-0030-4147-82df-6dc157679325'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:27.894401 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-81f2e3b3-0030-4147-82df-6dc157679325 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60a5ccc373a848428d3aac1632d86a93', 'request_id': 'req-81f2e3b3-0030-4147-82df-6dc157679325'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:27.900977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-81f2e3b3-0030-4147-82df-6dc157679325 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60a5ccc373a848428d3aac1632d86a93', 'request_id': 'req-81f2e3b3-0030-4147-82df-6dc157679325'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:28.001778 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-81f2e3b3-0030-4147-82df-6dc157679325 admin admin] The token cache is being invalidate because project 60a5ccc373a848428d3aac1632d86a93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:46:28.002067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-81f2e3b3-0030-4147-82df-6dc157679325 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:28.002673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-81f2e3b3-0030-4147-82df-6dc157679325 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:46:28.004128 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1426/2861] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:46:27 2022] DELETE /identity/v3/projects/60a5ccc373a848428d3aac1632d86a93 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:46:28.009680 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b80537-f8de-4a49-a995-208e44429b7d None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:28.010152 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b80537-f8de-4a49-a995-208e44429b7d None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:28.010266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b80537-f8de-4a49-a995-208e44429b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:28.029946 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-49b80537-f8de-4a49-a995-208e44429b7d None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:46:28.031081 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-49b80537-f8de-4a49-a995-208e44429b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:28.091038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1436/2862] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:46:28.098894 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1938fb-96cf-471f-9c08-3a4c70c775a9 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:28.099181 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1938fb-96cf-471f-9c08-3a4c70c775a9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:28.099465 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1938fb-96cf-471f-9c08-3a4c70c775a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:28.119338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-1a1938fb-96cf-471f-9c08-3a4c70c775a9 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:46:28.120543 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1a1938fb-96cf-471f-9c08-3a4c70c775a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:28.128409 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1427/2863] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:46:28.132800 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b80537-f8de-4a49-a995-208e44429b7d None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:28.143182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b80537-f8de-4a49-a995-208e44429b7d None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:28.147857 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521a4ac-8c52-489d-950d-d4eb83d7b7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:28.148699 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521a4ac-8c52-489d-950d-d4eb83d7b7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:28.148979 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521a4ac-8c52-489d-950d-d4eb83d7b7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:28.149282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521a4ac-8c52-489d-950d-d4eb83d7b7c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:28.149925 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0521a4ac-8c52-489d-950d-d4eb83d7b7c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81563) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:46:28.150254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0521a4ac-8c52-489d-950d-d4eb83d7b7c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:28.151852 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0521a4ac-8c52-489d-950d-d4eb83d7b7c3 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:28.163561 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1437/2864] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:46:28.168274 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1938fb-96cf-471f-9c08-3a4c70c775a9 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:28.177541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1938fb-96cf-471f-9c08-3a4c70c775a9 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:28.182851 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7df845-5ec7-4278-8034-2789709b20d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:28.183445 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7df845-5ec7-4278-8034-2789709b20d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:28.183723 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7df845-5ec7-4278-8034-2789709b20d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:28.183995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7df845-5ec7-4278-8034-2789709b20d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:28.184632 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7df845-5ec7-4278-8034-2789709b20d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:28.186579 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7df845-5ec7-4278-8034-2789709b20d5 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:28.267698 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1428/2865] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:46:28 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:46:28.272785 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521a4ac-8c52-489d-950d-d4eb83d7b7c3 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:28.284621 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521a4ac-8c52-489d-950d-d4eb83d7b7c3 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:28.309577 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:46:28.340340 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24327dc-ca05-4c9e-ae84-6d71db84108c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:28.340938 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24327dc-ca05-4c9e-ae84-6d71db84108c admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:28.341242 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24327dc-ca05-4c9e-ae84-6d71db84108c admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:28.341508 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24327dc-ca05-4c9e-ae84-6d71db84108c admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:28.342123 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24327dc-ca05-4c9e-ae84-6d71db84108c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:28.344095 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24327dc-ca05-4c9e-ae84-6d71db84108c admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:28.420387 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1438/2866] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:46:28 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 10:46:28.423815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7df845-5ec7-4278-8034-2789709b20d5 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:28.434600 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7df845-5ec7-4278-8034-2789709b20d5 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:28.439202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348552e-b6ce-4aa3-be83-7d20b923362a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:28.439798 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348552e-b6ce-4aa3-be83-7d20b923362a admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:28.440095 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348552e-b6ce-4aa3-be83-7d20b923362a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:28.440352 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348552e-b6ce-4aa3-be83-7d20b923362a admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:28.440988 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348552e-b6ce-4aa3-be83-7d20b923362a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:28.442603 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348552e-b6ce-4aa3-be83-7d20b923362a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:28.452276 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1429/2867] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:46:28.456062 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24327dc-ca05-4c9e-ae84-6d71db84108c admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:28.465793 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24327dc-ca05-4c9e-ae84-6d71db84108c admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:28.470416 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c901437-c565-45b9-936e-429d33e6bff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:28.474290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c901437-c565-45b9-936e-429d33e6bff4 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:28.474290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c901437-c565-45b9-936e-429d33e6bff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:28.474290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c901437-c565-45b9-936e-429d33e6bff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:28.474290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c901437-c565-45b9-936e-429d33e6bff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:28.474290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c901437-c565-45b9-936e-429d33e6bff4 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:28.484586 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1439/2868] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:46:28.488629 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348552e-b6ce-4aa3-be83-7d20b923362a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:28.499550 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348552e-b6ce-4aa3-be83-7d20b923362a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:28.504119 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08634ef0-199c-40ac-8767-a91abfaf8762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:28.504774 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08634ef0-199c-40ac-8767-a91abfaf8762 admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:28.505051 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08634ef0-199c-40ac-8767-a91abfaf8762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:28.505351 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08634ef0-199c-40ac-8767-a91abfaf8762 admin admin] PATH_INFO: `/v3/projects/311e428360814eee8331ecb4e8480db2/users/87314ad8a589480286d768fde7cb4068/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:28.519571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08634ef0-199c-40ac-8767-a91abfaf8762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=311e428360814eee8331ecb4e8480db2, user_id=87314ad8a589480286d768fde7cb4068, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:28.522209 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08634ef0-199c-40ac-8767-a91abfaf8762 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:28.614453 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1430/2869] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:46:28 2022] PUT /identity/v3/projects/311e428360814eee8331ecb4e8480db2/users/87314ad8a589480286d768fde7cb4068/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:46:28.619985 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60ca41-e6ee-4a64-ba10-c0745625a290 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:28.620266 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60ca41-e6ee-4a64-ba10-c0745625a290 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:28.620544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60ca41-e6ee-4a64-ba10-c0745625a290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:28.639836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-6c60ca41-e6ee-4a64-ba10-c0745625a290 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:46:28.641027 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-6c60ca41-e6ee-4a64-ba10-c0745625a290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:28.700393 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1440/2870] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 10:46:28.710175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08634ef0-199c-40ac-8767-a91abfaf8762 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:28.711902 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-08634ef0-199c-40ac-8767-a91abfaf8762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:28.731927 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:46:28.767973 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08634ef0-199c-40ac-8767-a91abfaf8762 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:28.771626 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4ac1a-8461-484e-b86b-71baf26b89c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:28.772218 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4ac1a-8461-484e-b86b-71baf26b89c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:28.772498 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4ac1a-8461-484e-b86b-71baf26b89c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:28.772780 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4ac1a-8461-484e-b86b-71baf26b89c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:28.781995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4ac1a-8461-484e-b86b-71baf26b89c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:28.783406 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4ac1a-8461-484e-b86b-71baf26b89c5 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:28.792690 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1431/2871] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:46:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:46:34.117324 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04c40d-98b9-423a-a2d4-fa149bb68d8a None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:34.117791 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04c40d-98b9-423a-a2d4-fa149bb68d8a None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:34.117913 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04c40d-98b9-423a-a2d4-fa149bb68d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:34.147107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-ca04c40d-98b9-423a-a2d4-fa149bb68d8a None None] MFA Rules not processed for user `87314ad8a589480286d768fde7cb4068`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:46:34.148863 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-ca04c40d-98b9-423a-a2d4-fa149bb68d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:34.217199 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1441/2872] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:46:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:46:34.221966 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4ac1a-8461-484e-b86b-71baf26b89c5 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:34.232354 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4ac1a-8461-484e-b86b-71baf26b89c5 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:34.236930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6b687-d5ed-4b94-8663-33a6f4c8d3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:34.237614 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6b687-d5ed-4b94-8663-33a6f4c8d3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:34.237896 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6b687-d5ed-4b94-8663-33a6f4c8d3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:34.238172 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6b687-d5ed-4b94-8663-33a6f4c8d3c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:34.238817 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6b687-d5ed-4b94-8663-33a6f4c8d3c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:34.240696 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6b687-d5ed-4b94-8663-33a6f4c8d3c2 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:34.347306 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1432/2873] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:46:34 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:46:34.351267 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04c40d-98b9-423a-a2d4-fa149bb68d8a None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:34.361680 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04c40d-98b9-423a-a2d4-fa149bb68d8a None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:34.395117 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:46:34.419902 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63931935-5b6b-4c29-89c8-9c92c7913e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:34.420527 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63931935-5b6b-4c29-89c8-9c92c7913e26 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:34.420807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63931935-5b6b-4c29-89c8-9c92c7913e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:34.421197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63931935-5b6b-4c29-89c8-9c92c7913e26 admin admin] PATH_INFO: `/v3/users` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:34.421786 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63931935-5b6b-4c29-89c8-9c92c7913e26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:34.423739 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63931935-5b6b-4c29-89c8-9c92c7913e26 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:34.523869 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1442/2874] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:46:34 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:46:34.527981 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6b687-d5ed-4b94-8663-33a6f4c8d3c2 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:34.538405 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6b687-d5ed-4b94-8663-33a6f4c8d3c2 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:34.543187 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba959f-e924-4568-b06d-0413b10f1487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:34.543783 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba959f-e924-4568-b06d-0413b10f1487 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:34.544070 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba959f-e924-4568-b06d-0413b10f1487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:34.544407 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba959f-e924-4568-b06d-0413b10f1487 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:34.545048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba959f-e924-4568-b06d-0413b10f1487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:34.546649 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba959f-e924-4568-b06d-0413b10f1487 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:34.557575 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1433/2875] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:46:34.561811 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63931935-5b6b-4c29-89c8-9c92c7913e26 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:34.571721 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63931935-5b6b-4c29-89c8-9c92c7913e26 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:34.576888 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04be1e-ff5d-45aa-8e6d-04dc2162e045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:34.577566 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04be1e-ff5d-45aa-8e6d-04dc2162e045 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:34.577846 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04be1e-ff5d-45aa-8e6d-04dc2162e045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:34.578117 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04be1e-ff5d-45aa-8e6d-04dc2162e045 admin admin] PATH_INFO: `/v3/projects/c0c11decc56d4573abe15a1b3c86bb75/users/90335266364f4531aaa92670b9cb6988/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:34.592387 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04be1e-ff5d-45aa-8e6d-04dc2162e045 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0c11decc56d4573abe15a1b3c86bb75, user_id=90335266364f4531aaa92670b9cb6988, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:34.595051 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04be1e-ff5d-45aa-8e6d-04dc2162e045 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:34.666256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1443/2876] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:46:34 2022] PUT /identity/v3/projects/c0c11decc56d4573abe15a1b3c86bb75/users/90335266364f4531aaa92670b9cb6988/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:46:34.670368 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba959f-e924-4568-b06d-0413b10f1487 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:34.680721 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba959f-e924-4568-b06d-0413b10f1487 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:34.745096 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badedd2a-4d9c-44a9-9f21-7cbc24d220e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:34.746078 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badedd2a-4d9c-44a9-9f21-7cbc24d220e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:34.746361 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badedd2a-4d9c-44a9-9f21-7cbc24d220e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:34.746634 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badedd2a-4d9c-44a9-9f21-7cbc24d220e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:34.747279 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badedd2a-4d9c-44a9-9f21-7cbc24d220e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:34.748875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badedd2a-4d9c-44a9-9f21-7cbc24d220e6 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:34.759581 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1434/2877] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:46:34.764562 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04be1e-ff5d-45aa-8e6d-04dc2162e045 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:34.775352 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04be1e-ff5d-45aa-8e6d-04dc2162e045 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:34.780004 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e2c77-6416-4379-b46b-79778a1f2a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:34.780658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e2c77-6416-4379-b46b-79778a1f2a28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:34.780938 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e2c77-6416-4379-b46b-79778a1f2a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:34.781256 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e2c77-6416-4379-b46b-79778a1f2a28 admin admin] PATH_INFO: `/v3/projects/c0c11decc56d4573abe15a1b3c86bb75/users/90335266364f4531aaa92670b9cb6988/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:34.783162 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e2c77-6416-4379-b46b-79778a1f2a28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0c11decc56d4573abe15a1b3c86bb75, user_id=90335266364f4531aaa92670b9cb6988, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:34.785869 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e2c77-6416-4379-b46b-79778a1f2a28 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:34.796824 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1444/2878] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:46:34 2022] PUT /identity/v3/projects/c0c11decc56d4573abe15a1b3c86bb75/users/90335266364f4531aaa92670b9cb6988/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 10:46:40.412514 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62548be-51f7-4c92-8a21-4461c936d295 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:40.413123 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62548be-51f7-4c92-8a21-4461c936d295 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:40.413123 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62548be-51f7-4c92-8a21-4461c936d295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:40.441908 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-f62548be-51f7-4c92-8a21-4461c936d295 None None] MFA Rules not processed for user `90335266364f4531aaa92670b9cb6988`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:46:40.443183 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f62548be-51f7-4c92-8a21-4461c936d295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:40.515659 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1435/2879] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:46:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:46:40.538966 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e2c77-6416-4379-b46b-79778a1f2a28 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:40.543022 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-0f6e2c77-6416-4379-b46b-79778a1f2a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:40.647240 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e2c77-6416-4379-b46b-79778a1f2a28 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:40.653665 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568c855-c1ed-412c-bb92-33796aed85c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:40.654273 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568c855-c1ed-412c-bb92-33796aed85c4 service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:40.654552 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568c855-c1ed-412c-bb92-33796aed85c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:40.654828 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568c855-c1ed-412c-bb92-33796aed85c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:40.713601 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568c855-c1ed-412c-bb92-33796aed85c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:46:40.715182 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568c855-c1ed-412c-bb92-33796aed85c4 service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:46:40.718772 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:46:40.724520 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1445/2880] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:46:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:46:43.217583 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a4a98-29df-4f18-8515-7d2cb3d5acda None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:43.218093 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a4a98-29df-4f18-8515-7d2cb3d5acda None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:43.218240 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a4a98-29df-4f18-8515-7d2cb3d5acda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:43.237481 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-622a4a98-29df-4f18-8515-7d2cb3d5acda None None] MFA Rules not processed for user `87314ad8a589480286d768fde7cb4068`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:46:43.238623 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-622a4a98-29df-4f18-8515-7d2cb3d5acda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:43.245807 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1436/2881] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:46:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:46:50.785501 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:46:53.232440 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=98,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:46:59.929727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568c855-c1ed-412c-bb92-33796aed85c4 service nova] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:46:59.931146 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-0568c855-c1ed-412c-bb92-33796aed85c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:46:59.990884 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568c855-c1ed-412c-bb92-33796aed85c4 service nova] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:46:59.994627 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea56a65-d8b5-4331-b587-5d4ac42d2a0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8a10f3332584a90a2231564527078cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:46:59.995292 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea56a65-d8b5-4331-b587-5d4ac42d2a0b service cinder] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:46:59.995571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea56a65-d8b5-4331-b587-5d4ac42d2a0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:46:59.995872 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea56a65-d8b5-4331-b587-5d4ac42d2a0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:46:59.997650 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4ea56a65-d8b5-4331-b587-5d4ac42d2a0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:47:00.010719 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea56a65-d8b5-4331-b587-5d4ac42d2a0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:47:00.012130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea56a65-d8b5-4331-b587-5d4ac42d2a0b service cinder] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:47:00.023011 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1446/2882] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:46:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:47:00.308431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a4a98-29df-4f18-8515-7d2cb3d5acda None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:47:00.310571 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-622a4a98-29df-4f18-8515-7d2cb3d5acda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:47:00.366856 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a4a98-29df-4f18-8515-7d2cb3d5acda None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:47:00.370293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed5808-9928-4141-ae15-dde1b32ae0a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:47:00.370953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed5808-9928-4141-ae15-dde1b32ae0a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:47:00.371231 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed5808-9928-4141-ae15-dde1b32ae0a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:47:00.371504 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed5808-9928-4141-ae15-dde1b32ae0a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:47:00.375713 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-8fed5808-9928-4141-ae15-dde1b32ae0a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:47:00.385450 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed5808-9928-4141-ae15-dde1b32ae0a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:47:00.386865 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed5808-9928-4141-ae15-dde1b32ae0a9 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:47:00.395051 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1437/2883] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:47:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:47:01.078351 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea56a65-d8b5-4331-b587-5d4ac42d2a0b service cinder] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:47:01.079881 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4ea56a65-d8b5-4331-b587-5d4ac42d2a0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:47:01.097900 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed5808-9928-4141-ae15-dde1b32ae0a9 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:47:01.106982 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed5808-9928-4141-ae15-dde1b32ae0a9 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:47:01.110749 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29a4dd-5245-4c76-8540-216638a410e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:47:01.111314 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29a4dd-5245-4c76-8540-216638a410e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:47:01.111590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29a4dd-5245-4c76-8540-216638a410e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:47:01.111863 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29a4dd-5245-4c76-8540-216638a410e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:47:01.113742 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-5e29a4dd-5245-4c76-8540-216638a410e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:47:01.127420 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29a4dd-5245-4c76-8540-216638a410e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:47:01.128821 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29a4dd-5245-4c76-8540-216638a410e9 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:47:01.140868 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1438/2884] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:47:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:47:01.150481 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea56a65-d8b5-4331-b587-5d4ac42d2a0b service cinder] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:47:01.156099 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c435390-8c38-4c53-b9ea-a0126490e007 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0ef47a60f54ca1853fd267e308c58b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:47:01.156716 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c435390-8c38-4c53-b9ea-a0126490e007 service placement] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:47:01.157028 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c435390-8c38-4c53-b9ea-a0126490e007 service placement] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:47:01.157353 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c435390-8c38-4c53-b9ea-a0126490e007 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:47:01.159091 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-8c435390-8c38-4c53-b9ea-a0126490e007 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:47:01.172405 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c435390-8c38-4c53-b9ea-a0126490e007 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:47:01.174021 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c435390-8c38-4c53-b9ea-a0126490e007 service placement] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:47:01.184779 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1447/2885] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:47:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:47:04.322047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29a4dd-5245-4c76-8540-216638a410e9 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:47:04.331763 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29a4dd-5245-4c76-8540-216638a410e9 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:47:04.335854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a55ad1-5b89-421b-9403-3fdd1aad60bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:47:04.336478 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a55ad1-5b89-421b-9403-3fdd1aad60bf service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:47:04.343227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a55ad1-5b89-421b-9403-3fdd1aad60bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:47:04.343227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a55ad1-5b89-421b-9403-3fdd1aad60bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:47:04.343227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-f2a55ad1-5b89-421b-9403-3fdd1aad60bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:47:04.350060 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a55ad1-5b89-421b-9403-3fdd1aad60bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:47:04.351488 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a55ad1-5b89-421b-9403-3fdd1aad60bf service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:47:04.360032 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1439/2886] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:47:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:47:05.201152 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c435390-8c38-4c53-b9ea-a0126490e007 service placement] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:47:05.223567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c435390-8c38-4c53-b9ea-a0126490e007 service placement] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:47:05.229309 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166dc6bf-6245-4d93-a04e-f819e4a36246 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:47:05.229925 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166dc6bf-6245-4d93-a04e-f819e4a36246 service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:47:05.230207 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166dc6bf-6245-4d93-a04e-f819e4a36246 service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:47:05.230483 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166dc6bf-6245-4d93-a04e-f819e4a36246 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:47:05.232220 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-166dc6bf-6245-4d93-a04e-f819e4a36246 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:47:05.247101 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166dc6bf-6245-4d93-a04e-f819e4a36246 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:47:05.248713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166dc6bf-6245-4d93-a04e-f819e4a36246 service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:47:05.260181 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1448/2887] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:47:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:47:14.356669 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=38 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:47:15.255663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=64 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:47:18.856969 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a55ad1-5b89-421b-9403-3fdd1aad60bf service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:47:18.866051 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a55ad1-5b89-421b-9403-3fdd1aad60bf service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:47:18.869660 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aba84c-7825-4e52-8122-7276225dde56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:47:18.870252 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aba84c-7825-4e52-8122-7276225dde56 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:47:18.870541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aba84c-7825-4e52-8122-7276225dde56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:47:18.870818 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aba84c-7825-4e52-8122-7276225dde56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:47:18.880144 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aba84c-7825-4e52-8122-7276225dde56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:47:18.881605 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aba84c-7825-4e52-8122-7276225dde56 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:47:18.890802 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1440/2888] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:47:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:47:28.886713 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=6 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:49:32.771178 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166dc6bf-6245-4d93-a04e-f819e4a36246 service nova] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:49:32.781337 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166dc6bf-6245-4d93-a04e-f819e4a36246 service nova] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:49:32.784932 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37241ac7-261f-4ba5-8933-0d6c22f93c36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:49:32.785556 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37241ac7-261f-4ba5-8933-0d6c22f93c36 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:49:32.785837 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37241ac7-261f-4ba5-8933-0d6c22f93c36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:49:32.786112 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37241ac7-261f-4ba5-8933-0d6c22f93c36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:49:32.787897 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-37241ac7-261f-4ba5-8933-0d6c22f93c36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:49:32.798749 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37241ac7-261f-4ba5-8933-0d6c22f93c36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:49:32.801196 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37241ac7-261f-4ba5-8933-0d6c22f93c36 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:49:32.813742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1449/2889] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:49:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:49:33.372047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aba84c-7825-4e52-8122-7276225dde56 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:49:33.380675 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aba84c-7825-4e52-8122-7276225dde56 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:49:33.384358 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d6da1-94c1-48a2-8325-e6224b1074b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:49:33.385012 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d6da1-94c1-48a2-8325-e6224b1074b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:49:33.385309 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d6da1-94c1-48a2-8325-e6224b1074b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:49:33.385583 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d6da1-94c1-48a2-8325-e6224b1074b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:49:33.387992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-524d6da1-94c1-48a2-8325-e6224b1074b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:49:33.399817 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d6da1-94c1-48a2-8325-e6224b1074b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:49:33.405036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d6da1-94c1-48a2-8325-e6224b1074b1 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:49:33.414625 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1441/2890] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:49:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 10:49:33.496764 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37241ac7-261f-4ba5-8933-0d6c22f93c36 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:49:33.505569 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37241ac7-261f-4ba5-8933-0d6c22f93c36 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:49:33.509234 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf77918-7b4c-4696-89db-6d7964ec51f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:49:33.509821 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf77918-7b4c-4696-89db-6d7964ec51f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:49:33.510093 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf77918-7b4c-4696-89db-6d7964ec51f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:49:33.510369 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf77918-7b4c-4696-89db-6d7964ec51f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:49:33.513037 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-cdf77918-7b4c-4696-89db-6d7964ec51f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:49:33.527740 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf77918-7b4c-4696-89db-6d7964ec51f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:49:33.529924 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf77918-7b4c-4696-89db-6d7964ec51f6 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:49:33.546540 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1450/2891] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:49:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:49:34.486854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d6da1-94c1-48a2-8325-e6224b1074b1 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:49:34.497622 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-524d6da1-94c1-48a2-8325-e6224b1074b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:49:34.508054 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d6da1-94c1-48a2-8325-e6224b1074b1 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:49:34.514458 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4faa4-7184-4e4a-b8da-673ffd892c01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:49:34.515149 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4faa4-7184-4e4a-b8da-673ffd892c01 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:49:34.515740 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4faa4-7184-4e4a-b8da-673ffd892c01 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:49:34.516025 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4faa4-7184-4e4a-b8da-673ffd892c01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:49:34.517828 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-26d4faa4-7184-4e4a-b8da-673ffd892c01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:49:34.543966 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4faa4-7184-4e4a-b8da-673ffd892c01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:49:34.543966 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4faa4-7184-4e4a-b8da-673ffd892c01 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:49:34.544922 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1442/2892] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:49:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:49:43.541217 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=12 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:49:44.539599 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=12 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:50:36.801829 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf77918-7b4c-4696-89db-6d7964ec51f6 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:50:36.803253 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-cdf77918-7b4c-4696-89db-6d7964ec51f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:50:36.818006 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf77918-7b4c-4696-89db-6d7964ec51f6 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:50:36.823231 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754582b5-e10b-4d90-b145-38c26c1ccd5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0ef47a60f54ca1853fd267e308c58b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:50:36.823842 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754582b5-e10b-4d90-b145-38c26c1ccd5f service placement] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:50:36.824122 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754582b5-e10b-4d90-b145-38c26c1ccd5f service placement] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:50:36.824396 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754582b5-e10b-4d90-b145-38c26c1ccd5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:50:36.826136 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-754582b5-e10b-4d90-b145-38c26c1ccd5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:50:36.837768 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754582b5-e10b-4d90-b145-38c26c1ccd5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:50:36.839325 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754582b5-e10b-4d90-b145-38c26c1ccd5f service placement] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:50:36.849624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1451/2893] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:50:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:50:46.844848 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=6 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:51:19.509319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4faa4-7184-4e4a-b8da-673ffd892c01 service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:19.520610 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4faa4-7184-4e4a-b8da-673ffd892c01 service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:19.526010 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea6b02-4d8a-4c0a-b5e0-3a4fe52eb751 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0ef47a60f54ca1853fd267e308c58b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:19.528327 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea6b02-4d8a-4c0a-b5e0-3a4fe52eb751 service placement] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:19.528327 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea6b02-4d8a-4c0a-b5e0-3a4fe52eb751 service placement] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:19.528327 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea6b02-4d8a-4c0a-b5e0-3a4fe52eb751 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:19.529558 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-eeea6b02-4d8a-4c0a-b5e0-3a4fe52eb751 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:51:19.541494 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea6b02-4d8a-4c0a-b5e0-3a4fe52eb751 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:19.543727 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea6b02-4d8a-4c0a-b5e0-3a4fe52eb751 service placement] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:19.554270 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1443/2894] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:51:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:51:23.138251 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754582b5-e10b-4d90-b145-38c26c1ccd5f service placement] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:23.149658 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754582b5-e10b-4d90-b145-38c26c1ccd5f service placement] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:23.156839 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9467f5f-e566-47d1-bd59-e4e0f35673f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0ef47a60f54ca1853fd267e308c58b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:23.157550 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9467f5f-e566-47d1-bd59-e4e0f35673f4 service placement] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:23.157906 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9467f5f-e566-47d1-bd59-e4e0f35673f4 service placement] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:23.158219 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9467f5f-e566-47d1-bd59-e4e0f35673f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:23.160612 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-b9467f5f-e566-47d1-bd59-e4e0f35673f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:51:23.173596 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9467f5f-e566-47d1-bd59-e4e0f35673f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:23.175363 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9467f5f-e566-47d1-bd59-e4e0f35673f4 service placement] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:23.186676 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1452/2895] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:51:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:51:23.262289 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea6b02-4d8a-4c0a-b5e0-3a4fe52eb751 service placement] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:23.332906 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea6b02-4d8a-4c0a-b5e0-3a4fe52eb751 service placement] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:23.337541 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05bca3-2023-4eab-acb3-c08e47ce1ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:23.338104 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05bca3-2023-4eab-acb3-c08e47ce1ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:23.338387 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05bca3-2023-4eab-acb3-c08e47ce1ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:23.338666 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05bca3-2023-4eab-acb3-c08e47ce1ece admin admin] PATH_INFO: `/v3/users/87314ad8a589480286d768fde7cb4068` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:23.339808 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05bca3-2023-4eab-acb3-c08e47ce1ece admin admin] RBAC: Authorizing `identity:delete_user(user_id=87314ad8a589480286d768fde7cb4068)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:23.341754 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05bca3-2023-4eab-acb3-c08e47ce1ece admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:23.541264 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3d05bca3-2023-4eab-acb3-c08e47ce1ece admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87314ad8a589480286d768fde7cb4068'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:23.541997 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3d05bca3-2023-4eab-acb3-c08e47ce1ece admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87314ad8a589480286d768fde7cb4068'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:23.613897 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3d05bca3-2023-4eab-acb3-c08e47ce1ece admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87314ad8a589480286d768fde7cb4068'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:23.614444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3d05bca3-2023-4eab-acb3-c08e47ce1ece admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87314ad8a589480286d768fde7cb4068'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:23.627065 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-3d05bca3-2023-4eab-acb3-c08e47ce1ece admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87314ad8a589480286d768fde7cb4068'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:23.640047 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1444/2896] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:51:23 2022] DELETE /identity/v3/users/87314ad8a589480286d768fde7cb4068 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:51:23.901750 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9467f5f-e566-47d1-bd59-e4e0f35673f4 service placement] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:23.912709 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9467f5f-e566-47d1-bd59-e4e0f35673f4 service placement] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:23.969807 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a793d-5090-48d1-806e-a899893e1f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:23.970542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a793d-5090-48d1-806e-a899893e1f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:23.970844 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a793d-5090-48d1-806e-a899893e1f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:23.971123 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a793d-5090-48d1-806e-a899893e1f98 admin admin] PATH_INFO: `/v3/projects/311e428360814eee8331ecb4e8480db2` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:23.972247 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a793d-5090-48d1-806e-a899893e1f98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=311e428360814eee8331ecb4e8480db2)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:23.974202 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a793d-5090-48d1-806e-a899893e1f98 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:24.078736 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7f0a793d-5090-48d1-806e-a899893e1f98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '311e428360814eee8331ecb4e8480db2', 'request_id': 'req-7f0a793d-5090-48d1-806e-a899893e1f98'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:24.084471 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7f0a793d-5090-48d1-806e-a899893e1f98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '311e428360814eee8331ecb4e8480db2', 'request_id': 'req-7f0a793d-5090-48d1-806e-a899893e1f98'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:24.090616 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7f0a793d-5090-48d1-806e-a899893e1f98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '311e428360814eee8331ecb4e8480db2', 'request_id': 'req-7f0a793d-5090-48d1-806e-a899893e1f98'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:24.173279 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7f0a793d-5090-48d1-806e-a899893e1f98 admin admin] The token cache is being invalidate because project 311e428360814eee8331ecb4e8480db2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:51:24.173948 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7f0a793d-5090-48d1-806e-a899893e1f98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:24.174287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7f0a793d-5090-48d1-806e-a899893e1f98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:24.175914 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1453/2897] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:51:23 2022] DELETE /identity/v3/projects/311e428360814eee8331ecb4e8480db2 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:51:24.179913 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05bca3-2023-4eab-acb3-c08e47ce1ece admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:24.181362 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3d05bca3-2023-4eab-acb3-c08e47ce1ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:51:24.194733 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05bca3-2023-4eab-acb3-c08e47ce1ece admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:24.249484 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3a280-2c12-49b4-88a7-85c3865909a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:24.250104 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3a280-2c12-49b4-88a7-85c3865909a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:24.250380 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3a280-2c12-49b4-88a7-85c3865909a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:24.250672 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3a280-2c12-49b4-88a7-85c3865909a9 admin admin] PATH_INFO: `/v3/users/90335266364f4531aaa92670b9cb6988` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:24.251787 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3a280-2c12-49b4-88a7-85c3865909a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90335266364f4531aaa92670b9cb6988)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:24.253717 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3a280-2c12-49b4-88a7-85c3865909a9 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:24.259491 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:51:24.417182 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ed3a280-2c12-49b4-88a7-85c3865909a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90335266364f4531aaa92670b9cb6988'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:24.418403 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ed3a280-2c12-49b4-88a7-85c3865909a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90335266364f4531aaa92670b9cb6988'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:24.482430 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ed3a280-2c12-49b4-88a7-85c3865909a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90335266364f4531aaa92670b9cb6988'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:24.483015 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ed3a280-2c12-49b4-88a7-85c3865909a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90335266364f4531aaa92670b9cb6988'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:24.497036 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-6ed3a280-2c12-49b4-88a7-85c3865909a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90335266364f4531aaa92670b9cb6988'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:24.509989 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1445/2898] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:51:24 2022] DELETE /identity/v3/users/90335266364f4531aaa92670b9cb6988 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:51:24.746535 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a793d-5090-48d1-806e-a899893e1f98 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:24.747967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-7f0a793d-5090-48d1-806e-a899893e1f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:51:24.765236 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a793d-5090-48d1-806e-a899893e1f98 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:24.822931 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063f288-cce7-4d9b-8173-a598f455badc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:24.823595 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063f288-cce7-4d9b-8173-a598f455badc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:24.823915 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063f288-cce7-4d9b-8173-a598f455badc admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:24.824255 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063f288-cce7-4d9b-8173-a598f455badc admin admin] PATH_INFO: `/v3/projects/c0c11decc56d4573abe15a1b3c86bb75` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:24.825497 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1063f288-cce7-4d9b-8173-a598f455badc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0c11decc56d4573abe15a1b3c86bb75)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:24.827643 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1063f288-cce7-4d9b-8173-a598f455badc admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:24.848727 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:51:24.917476 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-1063f288-cce7-4d9b-8173-a598f455badc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0c11decc56d4573abe15a1b3c86bb75', 'request_id': 'req-1063f288-cce7-4d9b-8173-a598f455badc'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:24.923450 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-1063f288-cce7-4d9b-8173-a598f455badc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0c11decc56d4573abe15a1b3c86bb75', 'request_id': 'req-1063f288-cce7-4d9b-8173-a598f455badc'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:24.929447 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-1063f288-cce7-4d9b-8173-a598f455badc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0c11decc56d4573abe15a1b3c86bb75', 'request_id': 'req-1063f288-cce7-4d9b-8173-a598f455badc'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:24.995617 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-1063f288-cce7-4d9b-8173-a598f455badc admin admin] The token cache is being invalidate because project c0c11decc56d4573abe15a1b3c86bb75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81563) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:51:24.995838 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-1063f288-cce7-4d9b-8173-a598f455badc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:24.996470 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-1063f288-cce7-4d9b-8173-a598f455badc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:51:24.997967 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1454/2899] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:51:24 2022] DELETE /identity/v3/projects/c0c11decc56d4573abe15a1b3c86bb75 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:51:25.002955 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2351431-b38a-4ed9-b0b8-e6c1a09355cc None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:25.003239 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2351431-b38a-4ed9-b0b8-e6c1a09355cc None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:25.005467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2351431-b38a-4ed9-b0b8-e6c1a09355cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:25.023714 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-a2351431-b38a-4ed9-b0b8-e6c1a09355cc None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:51:25.024917 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-a2351431-b38a-4ed9-b0b8-e6c1a09355cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:51:25.083505 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1446/2900] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:51:25.092809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2c9c7-5ab5-454f-a77e-bcccd5d32dbb None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:25.093123 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2c9c7-5ab5-454f-a77e-bcccd5d32dbb None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:25.093400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2c9c7-5ab5-454f-a77e-bcccd5d32dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:25.113035 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-92e2c9c7-5ab5-454f-a77e-bcccd5d32dbb None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:51:25.114248 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-92e2c9c7-5ab5-454f-a77e-bcccd5d32dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:51:25.122165 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1455/2901] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:51:25.126403 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2351431-b38a-4ed9-b0b8-e6c1a09355cc None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:25.136678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2351431-b38a-4ed9-b0b8-e6c1a09355cc None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:25.141391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4491c6c-5827-49ad-8d97-c3c477358a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:25.141992 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4491c6c-5827-49ad-8d97-c3c477358a4a admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:25.142305 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4491c6c-5827-49ad-8d97-c3c477358a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:25.142585 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4491c6c-5827-49ad-8d97-c3c477358a4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:25.143234 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4491c6c-5827-49ad-8d97-c3c477358a4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81562) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 10:51:25.143542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4491c6c-5827-49ad-8d97-c3c477358a4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:25.145208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4491c6c-5827-49ad-8d97-c3c477358a4a admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:25.158014 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1447/2902] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Sep 19 10:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 10:51:25.162234 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2c9c7-5ab5-454f-a77e-bcccd5d32dbb None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:25.171239 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2c9c7-5ab5-454f-a77e-bcccd5d32dbb None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:25.175835 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f21846-d408-4c94-b0d2-8b81c43ed7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:25.176416 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f21846-d408-4c94-b0d2-8b81c43ed7c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:25.176707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f21846-d408-4c94-b0d2-8b81c43ed7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:25.176976 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f21846-d408-4c94-b0d2-8b81c43ed7c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:25.177693 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f21846-d408-4c94-b0d2-8b81c43ed7c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:25.179571 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f21846-d408-4c94-b0d2-8b81c43ed7c1 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:25.439560 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1456/2903] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:51:25 2022] POST /identity/v3/projects => generated 379 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:51:25.443798 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4491c6c-5827-49ad-8d97-c3c477358a4a admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:25.455182 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4491c6c-5827-49ad-8d97-c3c477358a4a admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:25.510309 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5c71d-c0ae-4c20-9d03-32905a712e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:25.510896 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5c71d-c0ae-4c20-9d03-32905a712e94 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:25.511179 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5c71d-c0ae-4c20-9d03-32905a712e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:25.512245 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5c71d-c0ae-4c20-9d03-32905a712e94 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:25.512875 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd5c71d-c0ae-4c20-9d03-32905a712e94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:25.514825 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd5c71d-c0ae-4c20-9d03-32905a712e94 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:25.619890 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1448/2904] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:51:25 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:51:25.624248 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f21846-d408-4c94-b0d2-8b81c43ed7c1 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:25.634395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f21846-d408-4c94-b0d2-8b81c43ed7c1 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:25.639424 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d88f24-955e-4c38-a591-e32ff6edc42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:25.640011 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d88f24-955e-4c38-a591-e32ff6edc42e admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:25.640283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d88f24-955e-4c38-a591-e32ff6edc42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:25.640552 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d88f24-955e-4c38-a591-e32ff6edc42e admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:25.641254 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d88f24-955e-4c38-a591-e32ff6edc42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:25.642828 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d88f24-955e-4c38-a591-e32ff6edc42e admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:25.652661 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1457/2905] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:51:25.656427 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5c71d-c0ae-4c20-9d03-32905a712e94 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:25.661578 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:51:25.666770 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5c71d-c0ae-4c20-9d03-32905a712e94 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:25.671524 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91d8be-5b30-4f18-aec5-490d04429ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:25.672258 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91d8be-5b30-4f18-aec5-490d04429ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:25.672542 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91d8be-5b30-4f18-aec5-490d04429ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:25.672820 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91d8be-5b30-4f18-aec5-490d04429ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:25.673495 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91d8be-5b30-4f18-aec5-490d04429ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:25.675153 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91d8be-5b30-4f18-aec5-490d04429ec8 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:25.686183 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1449/2906] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:51:25.690287 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d88f24-955e-4c38-a591-e32ff6edc42e admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:25.700555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d88f24-955e-4c38-a591-e32ff6edc42e admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:25.705494 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb3615-9153-405d-a56e-72669325b5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:25.706141 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb3615-9153-405d-a56e-72669325b5bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:25.706412 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb3615-9153-405d-a56e-72669325b5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:25.706682 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb3615-9153-405d-a56e-72669325b5bf admin admin] PATH_INFO: `/v3/projects/83720c34e40e4f2ba7f9fe719db3c259/users/78b9e41066d04492bed776a5c013346f/roles/c44d082d88894b888e94797acf7655b7` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:25.721224 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb3615-9153-405d-a56e-72669325b5bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=83720c34e40e4f2ba7f9fe719db3c259, user_id=78b9e41066d04492bed776a5c013346f, role_id=c44d082d88894b888e94797acf7655b7)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:25.724070 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb3615-9153-405d-a56e-72669325b5bf admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:25.826454 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1458/2907] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:51:25 2022] PUT /identity/v3/projects/83720c34e40e4f2ba7f9fe719db3c259/users/78b9e41066d04492bed776a5c013346f/roles/c44d082d88894b888e94797acf7655b7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:51:25.831513 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56028bb5-32b3-4d9f-ab48-03da997e69d1 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:25.831809 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56028bb5-32b3-4d9f-ab48-03da997e69d1 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:25.832086 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56028bb5-32b3-4d9f-ab48-03da997e69d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:25.850999 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-56028bb5-32b3-4d9f-ab48-03da997e69d1 None None] MFA Rules not processed for user `c847b76e4e6649e18dc342c5d998066b`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:51:25.852227 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-56028bb5-32b3-4d9f-ab48-03da997e69d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:51:25.911678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1450/2908] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:51:25.921735 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb3615-9153-405d-a56e-72669325b5bf admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:25.923442 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-4bfb3615-9153-405d-a56e-72669325b5bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:51:25.978500 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb3615-9153-405d-a56e-72669325b5bf admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:25.982516 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d358c46-fe85-4dd9-98dd-db7ee2b18263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:25.983107 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d358c46-fe85-4dd9-98dd-db7ee2b18263 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:25.983382 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d358c46-fe85-4dd9-98dd-db7ee2b18263 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:25.983653 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d358c46-fe85-4dd9-98dd-db7ee2b18263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:25.992481 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d358c46-fe85-4dd9-98dd-db7ee2b18263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:25.993924 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d358c46-fe85-4dd9-98dd-db7ee2b18263 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:26.003707 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1459/2909] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:51:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:51:31.278977 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af3ce2-ca51-4bfd-89c7-19d7bd0100f9 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:31.279469 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af3ce2-ca51-4bfd-89c7-19d7bd0100f9 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:31.279592 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af3ce2-ca51-4bfd-89c7-19d7bd0100f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:31.308728 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-35af3ce2-ca51-4bfd-89c7-19d7bd0100f9 None None] MFA Rules not processed for user `78b9e41066d04492bed776a5c013346f`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:51:31.310286 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-35af3ce2-ca51-4bfd-89c7-19d7bd0100f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:51:31.390857 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1451/2910] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:51:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:51:31.395403 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d358c46-fe85-4dd9-98dd-db7ee2b18263 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:31.406850 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d358c46-fe85-4dd9-98dd-db7ee2b18263 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:31.411918 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e4043-0acc-4803-aa68-4c2de1555534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:31.412659 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e4043-0acc-4803-aa68-4c2de1555534 admin admin] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:31.413197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e4043-0acc-4803-aa68-4c2de1555534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:31.413518 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e4043-0acc-4803-aa68-4c2de1555534 admin admin] PATH_INFO: `/v3/projects` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:31.414216 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e4043-0acc-4803-aa68-4c2de1555534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:31.416382 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e4043-0acc-4803-aa68-4c2de1555534 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:31.502551 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1460/2911] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Sep 19 10:51:31 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:51:31.506978 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af3ce2-ca51-4bfd-89c7-19d7bd0100f9 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:31.519235 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af3ce2-ca51-4bfd-89c7-19d7bd0100f9 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:31.582440 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2a6c1-d080-4778-8a96-49c99563d283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:31.583110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2a6c1-d080-4778-8a96-49c99563d283 admin admin] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:31.583387 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2a6c1-d080-4778-8a96-49c99563d283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:31.583656 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2a6c1-d080-4778-8a96-49c99563d283 admin admin] PATH_INFO: `/v3/users` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:31.584445 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2a6c1-d080-4778-8a96-49c99563d283 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:31.586499 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2a6c1-d080-4778-8a96-49c99563d283 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:31.696900 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1452/2912] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Sep 19 10:51:31 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 10:51:31.701308 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e4043-0acc-4803-aa68-4c2de1555534 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:31.711613 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e4043-0acc-4803-aa68-4c2de1555534 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:31.716214 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417bd63-1029-42a3-8443-655d49ab3d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:31.716811 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417bd63-1029-42a3-8443-655d49ab3d51 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:31.717116 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417bd63-1029-42a3-8443-655d49ab3d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:31.717390 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417bd63-1029-42a3-8443-655d49ab3d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:31.718071 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417bd63-1029-42a3-8443-655d49ab3d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:31.719646 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417bd63-1029-42a3-8443-655d49ab3d51 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:31.730057 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1461/2913] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 10:51:31.734183 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2a6c1-d080-4778-8a96-49c99563d283 admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:31.744057 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2a6c1-d080-4778-8a96-49c99563d283 admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:31.749218 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319543b-f6e8-46d4-8af3-fa57b65885ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:31.749859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319543b-f6e8-46d4-8af3-fa57b65885ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:31.750140 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319543b-f6e8-46d4-8af3-fa57b65885ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:31.750413 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319543b-f6e8-46d4-8af3-fa57b65885ad admin admin] PATH_INFO: `/v3/projects/76e331f316984e83b4e0b757444fe8cc/users/d841191f96fd4767a0389cf957e036dd/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:31.758912 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:51:31.766660 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319543b-f6e8-46d4-8af3-fa57b65885ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=76e331f316984e83b4e0b757444fe8cc, user_id=d841191f96fd4767a0389cf957e036dd, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:31.769645 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319543b-f6e8-46d4-8af3-fa57b65885ad admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:31.832379 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1453/2914] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:51:31 2022] PUT /identity/v3/projects/76e331f316984e83b4e0b757444fe8cc/users/d841191f96fd4767a0389cf957e036dd/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:51:31.836669 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417bd63-1029-42a3-8443-655d49ab3d51 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:31.846724 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417bd63-1029-42a3-8443-655d49ab3d51 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:31.856038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:51:31.903400 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ecd2b-5d28-4b9f-97cb-f69c22e9afb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:31.904068 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ecd2b-5d28-4b9f-97cb-f69c22e9afb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:31.904350 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ecd2b-5d28-4b9f-97cb-f69c22e9afb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:31.904628 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ecd2b-5d28-4b9f-97cb-f69c22e9afb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:31.905308 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ecd2b-5d28-4b9f-97cb-f69c22e9afb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:31.906904 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ecd2b-5d28-4b9f-97cb-f69c22e9afb6 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:31.916532 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1462/2915] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Sep 19 10:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 10:51:31.921205 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319543b-f6e8-46d4-8af3-fa57b65885ad admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:31.931732 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319543b-f6e8-46d4-8af3-fa57b65885ad admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:31.936443 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c78318-10af-465b-8b51-0e185cb24e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:31.937110 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c78318-10af-465b-8b51-0e185cb24e7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:31.937480 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c78318-10af-465b-8b51-0e185cb24e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:31.937827 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c78318-10af-465b-8b51-0e185cb24e7d admin admin] PATH_INFO: `/v3/projects/76e331f316984e83b4e0b757444fe8cc/users/d841191f96fd4767a0389cf957e036dd/roles/1135c6b2ed2f4ba290fbb4a363456a49` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:31.939758 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c78318-10af-465b-8b51-0e185cb24e7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=76e331f316984e83b4e0b757444fe8cc, user_id=d841191f96fd4767a0389cf957e036dd, role_id=1135c6b2ed2f4ba290fbb4a363456a49)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:31.942464 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c78318-10af-465b-8b51-0e185cb24e7d admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:31.953282 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1454/2916] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Sep 19 10:51:31 2022] PUT /identity/v3/projects/76e331f316984e83b4e0b757444fe8cc/users/d841191f96fd4767a0389cf957e036dd/roles/1135c6b2ed2f4ba290fbb4a363456a49 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:51:37.270444 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dfd96-e5e2-4f19-a198-1e8a00510f6f None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:37.271498 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dfd96-e5e2-4f19-a198-1e8a00510f6f None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:37.271963 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dfd96-e5e2-4f19-a198-1e8a00510f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:37.311184 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-954dfd96-e5e2-4f19-a198-1e8a00510f6f None None] MFA Rules not processed for user `d841191f96fd4767a0389cf957e036dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:51:37.312513 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-954dfd96-e5e2-4f19-a198-1e8a00510f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:51:37.406006 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1463/2917] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:51:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:51:37.423840 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c78318-10af-465b-8b51-0e185cb24e7d admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:51:37.425721 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-92c78318-10af-465b-8b51-0e185cb24e7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:51:37.516798 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c78318-10af-465b-8b51-0e185cb24e7d admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:51:37.524906 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc75c9-d9a5-45e3-b984-a5d1e9bc2692 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:51:37.525576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc75c9-d9a5-45e3-b984-a5d1e9bc2692 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:37.525916 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc75c9-d9a5-45e3-b984-a5d1e9bc2692 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:37.526198 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc75c9-d9a5-45e3-b984-a5d1e9bc2692 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:37.606952 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc75c9-d9a5-45e3-b984-a5d1e9bc2692 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:51:37.608576 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc75c9-d9a5-45e3-b984-a5d1e9bc2692 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:51:37.618467 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1455/2918] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:51:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:51:47.402334 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=57 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:51:47.613938 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=66,keystone:INSERT=2 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:51:58.052197 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81909f9e-b08e-4afa-9acc-684d7bdec7b2 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:51:58.052737 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81909f9e-b08e-4afa-9acc-684d7bdec7b2 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:51:58.052853 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81909f9e-b08e-4afa-9acc-684d7bdec7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:51:58.071389 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-81909f9e-b08e-4afa-9acc-684d7bdec7b2 None None] MFA Rules not processed for user `78b9e41066d04492bed776a5c013346f`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:51:58.077713 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-81909f9e-b08e-4afa-9acc-684d7bdec7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:51:58.079836 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1464/2919] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:51:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:52:00.173353 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc75c9-d9a5-45e3-b984-a5d1e9bc2692 service nova] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:52:00.230862 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc75c9-d9a5-45e3-b984-a5d1e9bc2692 service nova] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:52:00.234592 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11cc34a-bc9d-4f8a-845f-3be79f3b188d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:52:00.235202 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11cc34a-bc9d-4f8a-845f-3be79f3b188d service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:52:00.235481 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11cc34a-bc9d-4f8a-845f-3be79f3b188d service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:52:00.235754 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11cc34a-bc9d-4f8a-845f-3be79f3b188d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:52:00.237590 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-b11cc34a-bc9d-4f8a-845f-3be79f3b188d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:52:00.251492 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11cc34a-bc9d-4f8a-845f-3be79f3b188d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:52:00.252930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11cc34a-bc9d-4f8a-845f-3be79f3b188d service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:52:00.263926 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1456/2920] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:52:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:52:01.378005 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81909f9e-b08e-4afa-9acc-684d7bdec7b2 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:52:01.379445 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-81909f9e-b08e-4afa-9acc-684d7bdec7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:52:01.391689 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81909f9e-b08e-4afa-9acc-684d7bdec7b2 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:52:01.395265 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1187e-a067-4530-b998-2d08aded93c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:52:01.395859 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1187e-a067-4530-b998-2d08aded93c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:52:01.396137 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1187e-a067-4530-b998-2d08aded93c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:52:01.396407 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1187e-a067-4530-b998-2d08aded93c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:52:01.398410 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-63d1187e-a067-4530-b998-2d08aded93c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:52:01.412346 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1187e-a067-4530-b998-2d08aded93c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:52:01.413792 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1187e-a067-4530-b998-2d08aded93c1 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:52:01.424718 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1465/2921] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:52:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:52:04.269761 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11cc34a-bc9d-4f8a-845f-3be79f3b188d service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:52:04.278608 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11cc34a-bc9d-4f8a-845f-3be79f3b188d service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:52:04.282186 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219aaa5c-02cf-4ce1-b23d-53f56b63fe77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:52:04.282776 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219aaa5c-02cf-4ce1-b23d-53f56b63fe77 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:52:04.283052 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219aaa5c-02cf-4ce1-b23d-53f56b63fe77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:52:04.283322 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219aaa5c-02cf-4ce1-b23d-53f56b63fe77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:52:04.284994 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-219aaa5c-02cf-4ce1-b23d-53f56b63fe77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:52:04.295146 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219aaa5c-02cf-4ce1-b23d-53f56b63fe77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:52:04.296589 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219aaa5c-02cf-4ce1-b23d-53f56b63fe77 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:52:04.304910 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1457/2922] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:52:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:52:11.419742 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=9 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:52:14.301191 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=32 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:52:18.468979 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1187e-a067-4530-b998-2d08aded93c1 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:52:18.478630 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1187e-a067-4530-b998-2d08aded93c1 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:52:18.482178 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343559a-1966-4bb9-9638-3019f581bb1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:52:18.482776 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343559a-1966-4bb9-9638-3019f581bb1b service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:52:18.483044 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343559a-1966-4bb9-9638-3019f581bb1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:52:18.483313 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343559a-1966-4bb9-9638-3019f581bb1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:52:18.491962 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343559a-1966-4bb9-9638-3019f581bb1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:52:18.493408 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343559a-1966-4bb9-9638-3019f581bb1b service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:52:18.502529 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1466/2923] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:52:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:52:20.331122 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219aaa5c-02cf-4ce1-b23d-53f56b63fe77 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:52:20.332588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-219aaa5c-02cf-4ce1-b23d-53f56b63fe77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:52:20.348858 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219aaa5c-02cf-4ce1-b23d-53f56b63fe77 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:52:20.355609 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e139979-cf52-4da4-8144-cd0f77d85956 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:52:20.356212 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e139979-cf52-4da4-8144-cd0f77d85956 service nova] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:52:20.356493 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e139979-cf52-4da4-8144-cd0f77d85956 service nova] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:52:20.356772 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e139979-cf52-4da4-8144-cd0f77d85956 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:52:20.358588 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-1e139979-cf52-4da4-8144-cd0f77d85956 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:52:20.370797 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e139979-cf52-4da4-8144-cd0f77d85956 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:52:20.372371 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e139979-cf52-4da4-8144-cd0f77d85956 service nova] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:52:20.383000 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1458/2924] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:52:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:52:28.498486 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=6 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:52:30.378417 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=6 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:54:20.921090 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343559a-1966-4bb9-9638-3019f581bb1b service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:54:20.971792 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343559a-1966-4bb9-9638-3019f581bb1b service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:54:20.977570 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe21fe-06c5-449d-b367-72d30faf43fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:54:20.978235 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe21fe-06c5-449d-b367-72d30faf43fc service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:54:20.978517 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe21fe-06c5-449d-b367-72d30faf43fc service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:54:20.978809 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe21fe-06c5-449d-b367-72d30faf43fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:54:20.980711 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-31fe21fe-06c5-449d-b367-72d30faf43fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:54:21.021951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe21fe-06c5-449d-b367-72d30faf43fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:54:21.021951 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe21fe-06c5-449d-b367-72d30faf43fc service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:54:21.035028 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1467/2925] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:54:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:54:23.756292 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b9c7e-c5a3-4952-84e2-b22aba945262 None None] REQUEST_METHOD: `POST` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:54:23.756765 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b9c7e-c5a3-4952-84e2-b22aba945262 None None] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:54:23.756883 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b9c7e-c5a3-4952-84e2-b22aba945262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:54:23.775953 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.auth.core [None req-542b9c7e-c5a3-4952-84e2-b22aba945262 None None] MFA Rules not processed for user `78b9e41066d04492bed776a5c013346f`. Rule list: `[]` (Enabled: `True`). {{(pid=81562) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:54:23.777679 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-542b9c7e-c5a3-4952-84e2-b22aba945262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:54:23.784470 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1459/2926] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:54:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:54:24.101891 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe21fe-06c5-449d-b367-72d30faf43fc service nova] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:54:24.121126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe21fe-06c5-449d-b367-72d30faf43fc service nova] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:54:24.124664 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9edc82-2c5c-4e5c-a18b-e1ce03ce5cd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:54:24.125283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9edc82-2c5c-4e5c-a18b-e1ce03ce5cd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:54:24.125555 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9edc82-2c5c-4e5c-a18b-e1ce03ce5cd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:54:24.125820 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9edc82-2c5c-4e5c-a18b-e1ce03ce5cd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:54:24.127542 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-fc9edc82-2c5c-4e5c-a18b-e1ce03ce5cd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:54:24.149341 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9edc82-2c5c-4e5c-a18b-e1ce03ce5cd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:54:24.150738 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9edc82-2c5c-4e5c-a18b-e1ce03ce5cd8 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:54:24.160761 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1468/2927] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:54:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:54:25.854685 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b9c7e-c5a3-4952-84e2-b22aba945262 None None] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:54:25.856160 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-542b9c7e-c5a3-4952-84e2-b22aba945262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:54:25.925394 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b9c7e-c5a3-4952-84e2-b22aba945262 None None] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:54:25.930359 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a3ea1-c9bb-4b70-94d7-0ecf00816894 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0ef47a60f54ca1853fd267e308c58b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:54:25.930964 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a3ea1-c9bb-4b70-94d7-0ecf00816894 service placement] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:54:25.931242 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a3ea1-c9bb-4b70-94d7-0ecf00816894 service placement] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:54:25.931517 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a3ea1-c9bb-4b70-94d7-0ecf00816894 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:54:25.933250 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-119a3ea1-c9bb-4b70-94d7-0ecf00816894 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:54:25.945102 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a3ea1-c9bb-4b70-94d7-0ecf00816894 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:54:25.946682 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a3ea1-c9bb-4b70-94d7-0ecf00816894 service placement] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:54:25.957386 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1460/2928] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:54:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:54:34.157130 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=43 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:54:35.952787 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=35 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:54:44.077290 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9edc82-2c5c-4e5c-a18b-e1ce03ce5cd8 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:54:44.091430 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9edc82-2c5c-4e5c-a18b-e1ce03ce5cd8 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:54:44.096949 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00000748-20da-4596-ab67-d3900d41c050 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4007794a30247c4a318af62d05e3b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:54:44.097597 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00000748-20da-4596-ab67-d3900d41c050 service nova] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:54:44.097871 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00000748-20da-4596-ab67-d3900d41c050 service nova] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:54:44.098186 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00000748-20da-4596-ab67-d3900d41c050 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:54:44.099882 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-00000748-20da-4596-ab67-d3900d41c050 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:54:44.112986 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00000748-20da-4596-ab67-d3900d41c050 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:54:44.114585 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00000748-20da-4596-ab67-d3900d41c050 service nova] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:54:44.125260 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1469/2929] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:54:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:54:54.120716 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=6 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:55:42.747436 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a3ea1-c9bb-4b70-94d7-0ecf00816894 service placement] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:55:42.748860 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-119a3ea1-c9bb-4b70-94d7-0ecf00816894 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:55:42.784872 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a3ea1-c9bb-4b70-94d7-0ecf00816894 service placement] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:55:42.789993 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89364d01-3eff-4c53-893d-469df591357a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0ef47a60f54ca1853fd267e308c58b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:55:42.790592 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89364d01-3eff-4c53-893d-469df591357a service placement] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:55:42.790874 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89364d01-3eff-4c53-893d-469df591357a service placement] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:55:42.791151 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89364d01-3eff-4c53-893d-469df591357a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:55:42.792887 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-89364d01-3eff-4c53-893d-469df591357a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:55:42.804099 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89364d01-3eff-4c53-893d-469df591357a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:55:42.805699 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89364d01-3eff-4c53-893d-469df591357a service placement] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:55:42.815975 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1461/2930] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:55:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:55:52.811299 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=15 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:56:36.949983 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fe39d-e7f1-4185-967d-69ddd5b682c6 None None] REQUEST_METHOD: `POST` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:56:36.950496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fe39d-e7f1-4185-967d-69ddd5b682c6 None None] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:56:36.950496 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fe39d-e7f1-4185-967d-69ddd5b682c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:56:36.970282 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.auth.core [None req-5f7fe39d-e7f1-4185-967d-69ddd5b682c6 None None] MFA Rules not processed for user `78b9e41066d04492bed776a5c013346f`. Rule list: `[]` (Enabled: `True`). {{(pid=81563) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 10:56:36.971472 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-5f7fe39d-e7f1-4185-967d-69ddd5b682c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:56:36.978242 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1470/2931] 10.222.0.42 () {58 vars in 958 bytes} [Mon Sep 19 10:56:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 10:56:37.118048 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89364d01-3eff-4c53-893d-469df591357a service placement] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:56:37.127697 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89364d01-3eff-4c53-893d-469df591357a service placement] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:56:37.131208 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a17dc6-0ddb-4e7f-9bab-08fee74e237b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:56:37.131793 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a17dc6-0ddb-4e7f-9bab-08fee74e237b service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:56:37.132067 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a17dc6-0ddb-4e7f-9bab-08fee74e237b service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:56:37.132338 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a17dc6-0ddb-4e7f-9bab-08fee74e237b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:56:37.140995 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a17dc6-0ddb-4e7f-9bab-08fee74e237b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:56:37.142414 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a17dc6-0ddb-4e7f-9bab-08fee74e237b service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:56:37.151124 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1462/2932] 10.222.0.42 () {60 vars in 1396 bytes} [Mon Sep 19 10:56:37 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:56:37.340683 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7fe39d-e7f1-4185-967d-69ddd5b682c6 None None] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:56:37.350327 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7fe39d-e7f1-4185-967d-69ddd5b682c6 None None] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:56:37.353948 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f64a7-21d7-4b91-b7de-b3c56b60ac23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:56:37.357868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f64a7-21d7-4b91-b7de-b3c56b60ac23 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:56:37.357868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f64a7-21d7-4b91-b7de-b3c56b60ac23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:56:37.357868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f64a7-21d7-4b91-b7de-b3c56b60ac23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:56:37.357868 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-2f2f64a7-21d7-4b91-b7de-b3c56b60ac23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:56:37.366663 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2f64a7-21d7-4b91-b7de-b3c56b60ac23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:56:37.368080 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2f64a7-21d7-4b91-b7de-b3c56b60ac23 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:56:37.376567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1471/2933] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:56:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:56:47.147293 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=6 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:56:47.373126 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=9 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:57:40.408025 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a17dc6-0ddb-4e7f-9bab-08fee74e237b service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:57:40.417573 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a17dc6-0ddb-4e7f-9bab-08fee74e237b service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:57:40.421304 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e02cb-c486-4781-9dab-c4782291aa23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:57:40.421900 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e02cb-c486-4781-9dab-c4782291aa23 service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:57:40.422182 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e02cb-c486-4781-9dab-c4782291aa23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:57:40.422459 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e02cb-c486-4781-9dab-c4782291aa23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:57:40.429762 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e02cb-c486-4781-9dab-c4782291aa23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:57:40.431175 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e02cb-c486-4781-9dab-c4782291aa23 service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:57:40.439092 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1463/2934] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:57:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:57:50.435475 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=6 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:58:22.908778 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f64a7-21d7-4b91-b7de-b3c56b60ac23 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:58:22.919261 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f64a7-21d7-4b91-b7de-b3c56b60ac23 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:58:22.922946 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25a916-5b43-4984-af80-4c8480ea7cb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:58:22.923543 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25a916-5b43-4984-af80-4c8480ea7cb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:58:22.923820 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25a916-5b43-4984-af80-4c8480ea7cb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:58:22.924117 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25a916-5b43-4984-af80-4c8480ea7cb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:58:22.941846 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25a916-5b43-4984-af80-4c8480ea7cb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:58:22.943308 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25a916-5b43-4984-af80-4c8480ea7cb3 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:58:22.965544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1472/2935] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:58:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:58:24.146678 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e02cb-c486-4781-9dab-c4782291aa23 service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:58:24.155807 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e02cb-c486-4781-9dab-c4782291aa23 service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:58:24.159390 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73aa3a-1c6e-4cf0-9f38-136950bd1b7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:58:24.159972 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73aa3a-1c6e-4cf0-9f38-136950bd1b7e service neutron] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:58:24.160246 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73aa3a-1c6e-4cf0-9f38-136950bd1b7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:58:24.160516 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73aa3a-1c6e-4cf0-9f38-136950bd1b7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:58:24.170421 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73aa3a-1c6e-4cf0-9f38-136950bd1b7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:58:24.171815 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73aa3a-1c6e-4cf0-9f38-136950bd1b7e service neutron] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:58:24.182431 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1464/2936] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:58:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:58:32.962230 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=13 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:58:34.177656 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=6 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:58:38.469544 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25a916-5b43-4984-af80-4c8480ea7cb3 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:58:38.478421 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25a916-5b43-4984-af80-4c8480ea7cb3 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:58:38.482882 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ca6a0-40f1-4d01-9541-67a907533fc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a282e74d177544bbbfba0fdb5e52df60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:58:38.483478 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ca6a0-40f1-4d01-9541-67a907533fc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:58:38.483754 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ca6a0-40f1-4d01-9541-67a907533fc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:58:38.484024 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ca6a0-40f1-4d01-9541-67a907533fc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:58:38.551625 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ca6a0-40f1-4d01-9541-67a907533fc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:58:38.553120 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ca6a0-40f1-4d01-9541-67a907533fc4 service neutron] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:58:38.571533 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1473/2937] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:58:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:58:41.724273 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73aa3a-1c6e-4cf0-9f38-136950bd1b7e service neutron] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:58:41.735859 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73aa3a-1c6e-4cf0-9f38-136950bd1b7e service neutron] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:58:41.741186 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd026d-827f-4309-9267-4dbdd52bb8f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0ef47a60f54ca1853fd267e308c58b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:58:41.742186 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd026d-827f-4309-9267-4dbdd52bb8f1 service placement] REQUEST_METHOD: `GET` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:58:41.742796 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd026d-827f-4309-9267-4dbdd52bb8f1 service placement] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:58:41.743039 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd026d-827f-4309-9267-4dbdd52bb8f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:58:41.744807 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-3ebd026d-827f-4309-9267-4dbdd52bb8f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:58:41.757189 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd026d-827f-4309-9267-4dbdd52bb8f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:58:41.758771 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd026d-827f-4309-9267-4dbdd52bb8f1 service placement] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:58:41.769814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1465/2938] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 10:58:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 10:58:45.679311 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ca6a0-40f1-4d01-9541-67a907533fc4 service neutron] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:58:45.684899 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ca6a0-40f1-4d01-9541-67a907533fc4 service neutron] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:58:45.692655 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6614d8c-db99-4fab-bb64-4e3fb1134b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:58:45.693339 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6614d8c-db99-4fab-bb64-4e3fb1134b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:58:45.693622 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6614d8c-db99-4fab-bb64-4e3fb1134b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:58:45.693895 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6614d8c-db99-4fab-bb64-4e3fb1134b3d admin admin] PATH_INFO: `/v3/users/78b9e41066d04492bed776a5c013346f` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:58:45.695038 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6614d8c-db99-4fab-bb64-4e3fb1134b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b9e41066d04492bed776a5c013346f)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:58:45.697004 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6614d8c-db99-4fab-bb64-4e3fb1134b3d admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:58:45.938394 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-e6614d8c-db99-4fab-bb64-4e3fb1134b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b9e41066d04492bed776a5c013346f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:46.051774 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-e6614d8c-db99-4fab-bb64-4e3fb1134b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b9e41066d04492bed776a5c013346f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:46.052428 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-e6614d8c-db99-4fab-bb64-4e3fb1134b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b9e41066d04492bed776a5c013346f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:46.053043 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-e6614d8c-db99-4fab-bb64-4e3fb1134b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b9e41066d04492bed776a5c013346f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:46.061408 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-e6614d8c-db99-4fab-bb64-4e3fb1134b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b9e41066d04492bed776a5c013346f'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:46.079624 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1474/2939] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:58:45 2022] DELETE /identity/v3/users/78b9e41066d04492bed776a5c013346f => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:58:46.327391 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd026d-827f-4309-9267-4dbdd52bb8f1 service placement] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:58:46.338673 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd026d-827f-4309-9267-4dbdd52bb8f1 service placement] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:58:46.396857 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8b23f-3e39-426c-b16e-1daa8742a1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:58:46.397531 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8b23f-3e39-426c-b16e-1daa8742a1ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:58:46.397814 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8b23f-3e39-426c-b16e-1daa8742a1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:58:46.398092 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8b23f-3e39-426c-b16e-1daa8742a1ee admin admin] PATH_INFO: `/v3/projects/83720c34e40e4f2ba7f9fe719db3c259` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:58:46.399246 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a8b23f-3e39-426c-b16e-1daa8742a1ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=83720c34e40e4f2ba7f9fe719db3c259)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:58:46.401193 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a8b23f-3e39-426c-b16e-1daa8742a1ee admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:58:46.513981 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-21a8b23f-3e39-426c-b16e-1daa8742a1ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83720c34e40e4f2ba7f9fe719db3c259', 'request_id': 'req-21a8b23f-3e39-426c-b16e-1daa8742a1ee'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:46.519598 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-21a8b23f-3e39-426c-b16e-1daa8742a1ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83720c34e40e4f2ba7f9fe719db3c259', 'request_id': 'req-21a8b23f-3e39-426c-b16e-1daa8742a1ee'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:46.525548 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-21a8b23f-3e39-426c-b16e-1daa8742a1ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83720c34e40e4f2ba7f9fe719db3c259', 'request_id': 'req-21a8b23f-3e39-426c-b16e-1daa8742a1ee'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:46.603489 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-21a8b23f-3e39-426c-b16e-1daa8742a1ee admin admin] The token cache is being invalidate because project 83720c34e40e4f2ba7f9fe719db3c259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:58:46.603777 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-21a8b23f-3e39-426c-b16e-1daa8742a1ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:46.604882 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-21a8b23f-3e39-426c-b16e-1daa8742a1ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:46.606444 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1466/2940] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:58:46 2022] DELETE /identity/v3/projects/83720c34e40e4f2ba7f9fe719db3c259 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:58:46.610567 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6614d8c-db99-4fab-bb64-4e3fb1134b3d admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:58:46.612014 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-e6614d8c-db99-4fab-bb64-4e3fb1134b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:58:46.626013 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6614d8c-db99-4fab-bb64-4e3fb1134b3d admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:58:46.670411 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:58:46.682112 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ded5f-1969-4489-9b37-6aa4ec008216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:58:46.682813 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ded5f-1969-4489-9b37-6aa4ec008216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:58:46.683020 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ded5f-1969-4489-9b37-6aa4ec008216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:58:46.683291 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ded5f-1969-4489-9b37-6aa4ec008216 admin admin] PATH_INFO: `/v3/users/d841191f96fd4767a0389cf957e036dd` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:58:46.684415 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ded5f-1969-4489-9b37-6aa4ec008216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d841191f96fd4767a0389cf957e036dd)` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:58:46.686371 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ded5f-1969-4489-9b37-6aa4ec008216 admin admin] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:58:46.911099 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7d1ded5f-1969-4489-9b37-6aa4ec008216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd841191f96fd4767a0389cf957e036dd'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:46.999766 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7d1ded5f-1969-4489-9b37-6aa4ec008216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd841191f96fd4767a0389cf957e036dd'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:47.000395 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7d1ded5f-1969-4489-9b37-6aa4ec008216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd841191f96fd4767a0389cf957e036dd'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:47.001006 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7d1ded5f-1969-4489-9b37-6aa4ec008216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd841191f96fd4767a0389cf957e036dd'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:47.008619 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.notifications [None req-7d1ded5f-1969-4489-9b37-6aa4ec008216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd841191f96fd4767a0389cf957e036dd'} {{(pid=81563) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:47.026769 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1475/2941] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Sep 19 10:58:46 2022] DELETE /identity/v3/users/d841191f96fd4767a0389cf957e036dd => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:58:47.333291 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8b23f-3e39-426c-b16e-1daa8742a1ee admin admin] Authenticating user token {{(pid=81562) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 10:58:47.336319 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.fernet_utils [None req-21a8b23f-3e39-426c-b16e-1daa8742a1ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81562) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 10:58:47.349168 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8b23f-3e39-426c-b16e-1daa8742a1ee admin admin] Validating token access rules against request {{(pid=81562) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 10:58:47.415702 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da221a-5f10-4a3c-a9f3-7ce9062af662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c847b76e4e6649e18dc342c5d998066b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8dac977aca46d48c891cd4c281d8cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81562) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 10:58:47.416327 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da221a-5f10-4a3c-a9f3-7ce9062af662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 10:58:47.416610 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da221a-5f10-4a3c-a9f3-7ce9062af662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 10:58:47.417519 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da221a-5f10-4a3c-a9f3-7ce9062af662 admin admin] PATH_INFO: `/v3/projects/76e331f316984e83b4e0b757444fe8cc` {{(pid=81562) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 10:58:47.418700 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da221a-5f10-4a3c-a9f3-7ce9062af662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76e331f316984e83b4e0b757444fe8cc)` {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 10:58:47.420612 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da221a-5f10-4a3c-a9f3-7ce9062af662 admin admin] RBAC: Authorization granted {{(pid=81562) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 10:58:47.428641 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:58:47.570562 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-45da221a-5f10-4a3c-a9f3-7ce9062af662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76e331f316984e83b4e0b757444fe8cc', 'request_id': 'req-45da221a-5f10-4a3c-a9f3-7ce9062af662'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:47.575854 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-45da221a-5f10-4a3c-a9f3-7ce9062af662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76e331f316984e83b4e0b757444fe8cc', 'request_id': 'req-45da221a-5f10-4a3c-a9f3-7ce9062af662'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:47.582259 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-45da221a-5f10-4a3c-a9f3-7ce9062af662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76e331f316984e83b4e0b757444fe8cc', 'request_id': 'req-45da221a-5f10-4a3c-a9f3-7ce9062af662'} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:47.667930 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-45da221a-5f10-4a3c-a9f3-7ce9062af662 admin admin] The token cache is being invalidate because project 76e331f316984e83b4e0b757444fe8cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81562) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 10:58:47.668216 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-45da221a-5f10-4a3c-a9f3-7ce9062af662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:47.668833 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG keystone.notifications [None req-45da221a-5f10-4a3c-a9f3-7ce9062af662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81562) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 10:58:47.670330 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: [pid: 81562|app: 0|req: 1467/2942] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Sep 19 10:58:47 2022] DELETE /identity/v3/projects/76e331f316984e83b4e0b757444fe8cc => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 10:58:57.023177 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 10:58:57.664218 dv-cinder-858275-1-iscsi devstack@keystone.service[81562]: DEBUG dbcounter [-] [81562] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81562) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:00:48.730283 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ded5f-1969-4489-9b37-6aa4ec008216 admin admin] Authenticating user token {{(pid=81563) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:00:48.731674 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-7d1ded5f-1969-4489-9b37-6aa4ec008216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:00:48.800677 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ded5f-1969-4489-9b37-6aa4ec008216 admin admin] Validating token access rules against request {{(pid=81563) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:00:48.865464 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcc56a-198e-4846-9831-e92327ea593f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce0ef47a60f54ca1853fd267e308c58b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e45b8fbe79994e7585095328a7ec9260', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81563) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:00:48.866071 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcc56a-198e-4846-9831-e92327ea593f service placement] REQUEST_METHOD: `GET` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:00:48.866349 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcc56a-198e-4846-9831-e92327ea593f service placement] SCRIPT_NAME: `/identity` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:00:48.866616 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcc56a-198e-4846-9831-e92327ea593f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81563) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:00:48.868323 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.fernet_utils [None req-7bdcc56a-198e-4846-9831-e92327ea593f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81563) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:00:48.879935 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdcc56a-198e-4846-9831-e92327ea593f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:00:48.881522 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdcc56a-198e-4846-9831-e92327ea593f service placement] RBAC: Authorization granted {{(pid=81563) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:00:48.891490 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: [pid: 81563|app: 0|req: 1476/2943] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Sep 19 11:00:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:00:58.887024 dv-cinder-858275-1-iscsi devstack@keystone.service[81563]: DEBUG dbcounter [-] [81563] Writing DB stats keystone:SELECT=58 {{(pid=81563) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}